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Executive Summary 


This mixed-methods study examined four STEM communities (BioQUEST, Project Kaleidoscope, the POGIL 
Project, and SENCER) in order to better understand the roles of these communities in advancing the goals of scal¬ 
ing STEM education reform. The project explored three key questions: 

• How do members and leaders of communities of practice (CoPs * 1 ) perceive CoP design (membership, 
structure, communication, activities, and organization to support new knowledge development and ac¬ 
tion) shapes the ability to achieve goals (around undergraduate STEM pedagogical change and diffu¬ 
sion)? 

• What are the perceived benefits of participation in a STEM reform community of practice or network 
for the individual participants and for their campuses? 

• How do communities of practice and networks form, and how are they sustained in ways that help 
them to achieve their goals? 

The following are the key findings related to these questions: 

I lhe study identified a novel approach to improving STEM education, which 
we have called communities of transformation. 

This study identified a unique variant of communities of practice, called communities of transformation (CoTs) that 
are present in the STEM reform area. The defining feature of these newly identified entities is their focus on explor¬ 
ing philosophically, in deep and fundamental ways, how science is taught. This can lead to more substantive changes 
that have the potential to address the problems described in national reports around underrepresentation of women 
and underserved minorities, persistence rates, and success among students. These communities of transformation 
create innovative spaces that have the potential to shift institutional and disciplinary norms. We identify how these 
differ from more traditional professional development models, including campus-based professional development 
and disciplinary meetings. 

2 Communities of transformation address both individual lacuHy and 
bioadersystemic change. 

Much of the early work to improve STEM education has focused on altering individual faculty behavior through 
faculty development and dissemination of best practices. Some more recent efforts (e.g., Association of American 
Universities’ STEM reform initiative) focus on changing broader systemic and institutional norms. Communities of 
transformation provide support for individual faculty change, but they also simultaneously work (to varying degrees) 
to shift departmental cultures, institutional norms, and disciplinary values. In this report, we describe work that these 
communities conducted to alter the conversation around teaching within disciplines, as well as evidence of broader 
impact achieved through service on national committees and task forces aimed at improving STEM education. Their 
unique work to address both the individual faculty and the broader system is a compelling strategy for change. We 
found that the strength of these efforts lies in working from the ground up, with individual faculty buy-in, motiva¬ 
tion, and support for improving practice. Additionally, in general, institutional type, discipline, and rank/appointment 
status were not significantly associated with the outcomes we measured, when accounting for our other engagement, 
design, and motivation variables. We think this points to the potential for CoTs resembling those in our study to 
contribute to overcoming typical barriers to reform such as reward structures, disciplinary cultures, and a lack of 
institutional leadership. 

1 A community of practice is a group of people who share a concern or a passion for something they do and learn how to do 
it as they interact regularly (Allee, 2000; Lave, 1988; Wenger, 1998 and 2007). 
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3 Benefits of these communities accrue to both individual faculty and to 
their institutions. 

Participants reported that the greatest benefits of involvement in these communities came in the form of learning 
and improving in their teaching, reenergizing them in their sense of satisfaction and fulfillment in their work, and 
gaining credibility for their work related to STEM reform. Additionally, nearly 35% of participants indicated that 
engagement in these communities contributed to changes related to STEM reform in their departments, while 
more than one in five participants indicated that some sort of institutional change had come about as a result of 
involvement in these communities. We also identified how involvement of several individuals from a single institu¬ 
tion increased departmental and institutional benefits. In fact, one of the largest effect sizes we observed to predict 
departmental and institutional benefits came from having more peers from the same institution involved together 
in the community. In general, it is also important to recognize that the longer faculty remained involved with these 
groups, the more benefits they reported. 

4 Communities of transformation provide significant benefits for women 
faculty and forfaculty of color. 

For nearly all of the individual benefits we studied (except for involvement leading to publications), female faculty 
members reported statistically significantly greater benefits resulting from their participation in these communities 
than their male counterparts. Faculty of color indicated greater benefits than White faculty members in several key 
areas: networking, being afforded the opportunity to pursue new grants or projects, gaining credibility for their ap¬ 
proach to professional work, and gaining skills to make the transition from faculty work to administration. 



Positive outcomes follow from an engaging philosophy that is lived in 
programmatic activities and fostered through a supportive and mentoring 
community. 


Faculty report that they make gains, such as improved teaching, becoming leaders for change, and renewed enthusi¬ 
asm for their careers, as a result of the engaging philosophies that are part of these four communities of transforma¬ 
tion. Survey results and interviews both suggest that embedding the philosophy in events, workshops, newsletters, 
and other key communications made faculty better able to adopt new approaches to teaching/learning. Faculty also 
appreciated role modeling by leaders in these communities. The communities of transformation had a system for 
bringing new faculty along by first brainstorming changes and ideas with them, then helping them address chal¬ 
lenges on their campuses, and then providing more formal mentoring. Further, leadership that espouses and models 
the philosophy, fostering a supportive culture, further contributes to achieving outcomes. 


6 Communities of transformation follow similar trajectories as they evolve 
from an idea to a community. 

The lifecycles of these communities of transformation follow a similar trajectory that moves from showing potential 
(testing out initial ideas, obtaining initial grants, coming together for discussion over years of gestation), to coalesc¬ 
ing (naming the problem, forming cultures), to maturing (building communities, obtaining new grants, developing 
leadership), to stewardship (creating leadership succession plans, putting into place a viable financial model, hiring 
professional staff, creating and deploying a framework of research, feedback, and assessment, following a focused yet 
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flexible strategy). The importance of this similar trajectory demonstrates that future communities of transformation 
can follow the steps of these groups to successfully evolve and navigate challenges. 

7 Communities of transformation lace common challenges and must 
develop particular strategies to navigate them. 

A common set of challenges emerged that provides clear direction for future groups in terms of obstacles to antici¬ 
pate in their work. These challenges include: funding, shifting focus, community leadership too much identified with 
an individual leader, project-focused versus community-focused decisions, staleness, legitimacy, the dominant cul¬ 
ture of science education, maintaining community integrity, focus on general faculty improvement versus a specific 
pedagogical approach, and increasing and changing demands on faculty. This report articulates these challenges and 
offers advice for navigating them. 

8 Communities of tra nsfbrmation rely on a specific set of avenues for 
expanding impact 

In order to expand membership and impact, the communities studied took six different avenues, all of which show 
promise for use by future communities of transformation. We have categorized these avenues as disciplinary, insti¬ 
tutional, sector-focused, constituent-based, national, and international approaches. The study identified that com¬ 
munities can be more successful when they expand in areas where they have some existing strengths or assets. For 
example, two communities studied had connections to leaders in disciplinary societies, while two others had connec¬ 
tions to administrators to leverage for an institutional approach to expansion. This report also documents challenges 
for expansion. 


9 Future communities of tra nsfoimation can draw on the sustainability 
model identified and developed through this study. 

Since communities of practice are typically organic organizations that can come and go, the communities of transfor¬ 
mation engaged in the important work that was the focus of this study must have plans for sustainability. Such plans 
are critical to the expansion, success, and impact of communities of transformation. This report offers a sustainability 
model that includes the following elements: creating leadership succession plans, putting into place a viable financial 
model, hiring professional staff, creating and deploying a framework of research, feedback, and assessment, and fol¬ 
lowing a focused yet flexible strategy. 


10 ; 


re are further ways that communities of tra nsfoimation can extend their 
pact 


The study also identified some key ways that these communities of transformation can increase their already signifi¬ 
cant impact, through working with centers for teaching and learning on campuses, helping faculty create professional 
learning communities on their own campuses, inviting teams from campuses to maximize impact, and working even 
more deeply with disciplinary societies, among other recommendations. 
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I. Background and Overview of 
Project 


For the past 20 years, countless reports have been issued calling for reform of undergraduate STEM education to 
improve student learning and success for both majors and non-majors. Recent reports describe the need to focus on 
creating more student-centered learning environments that use the most effective research-based teaching, learn¬ 
ing, and assessment strategies (American Association for Advancement of Science, 2011; Howard Hughes Medical 
Institute, 2009; National Academies, 2010; National Science Foundation, 2010). All of these reports call attention 
to a set of problems in undergraduate STEM education: 1. Few students choose to be STEM majors; 2. Tradition¬ 
ally underrepresented groups have extremely low participation in STEM fields; 3. STEM majors face low graduation 
rates; and, 4. There are broad skills that graduates lack as they complete STEM majors (e.g., teamwork, writing) and 
non-STEM majors (e.g., quantitative reasoning, analytical thinking), making it difficult for them to meet workplace 
needs in our technology-based knowledge economy. While experts across the country generally agree on the nature 
of the problems and on some of the interventions needed, there is less agreement about how to create widespread 
change. Some emerging evidence suggests that current approaches are ineffective (Fairweather, 2009). 

Systemic change in higher education has proven difficult. Isolated efforts, such as funding short, one-time faculty 
innovations, have not been effective at yielding the kind of widespread change articulated in national reports. This 
is due to the fact that colleges and universities are complex systems in which multiple factors influence educators’ 
actions, values, and behaviors. Given the size and scale of higher education, changing individual faculty members or 
even isolated departments will have minimal impact. Fairweather (2009) notes, in his report to the National Acad¬ 
emies Research Council Board of Science Education, that the presumption that funding individual innovations will 
lead to widespread changes is spurious, and it is not born out by the evidence. Instead, he advocates for engaging 
institutional leaders on campus to overcome a set of existing hurdles, such as reward structures, and he notes the 
importance of professional networks as avenues to scale up change. He observes that networks systematically engage 
large numbers of faculty on an ongoing and sustained basis, which is more likely to lead to change. 

In a review article commissioned by the National Academies, Ann Austin outlines the factors that need to be ad¬ 
dressed in order to promulgate more evidence-based teaching practices. Among the top factors is professional de¬ 
velopment that involves communities of practice (CoPs) that “provide opportunities for faculty members to interact 
with others as they explore new assumptions and try out new approaches to teaching...in an environment that simul¬ 
taneously provides challenge and support.” Over the years, National Science Foundation (NSF) has funded networks 
and CoPs (see the important distinction between these below) as means for disseminating innovations and creating 
change; in more recent years, this has become an even more prominent strategy. 

Institutional, regional, and national CoPs and networks that are focused on providing knowledge, support, and exem¬ 
plary models for STEM education have been identified in reports as important vehicles for creating change, yet there 
is little systematic research on how to best structure them, nor are there any data about their outcomes or impact. 

The use of networks in such projects is motivated by solid social science research. Over the last fifty years researchers 
such as Everett Rogers have identified how social networks are the primary vehicles for the dissemination of innova¬ 
tions. Yet, this research on social networks has focused on changes quite different from those required by educational 
reform, and it has focused on other types of contexts, such as farming or medical practice 2 . More recent research 
on communities of practice identifies how social networks that foster conversation and learning within educational 
contexts are effective vehicles for peer professional development and adaptation of innovative strategies that result 
in change (Daly, 2010). However, while we know many such CoPs lead to change, we do not know how they can 
be best designed to achieve their goals. In particular, we have little information about STEM faculty networks, and 
about whether they require any unique design features to help stimulate reform. 
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In summary, while networking efforts have emerged as critical strategies for creating innovation in higher education, 
we know very little about such networks beyond the fact that they are positively linked to facilitating change. From 
the existing research, we know that CoPs and social networks offer certain advantages within change processes—for 
example, communication systems, knowledge transfer, and access to expertise (Tsai, 2002; Valente, 1995). Yet, having 
worked with several networks in STEM undergraduate education, we sense that there are other less well documented 
benefits, such as leadership development, that need to be identified and cultivated. Further, we also need to seek more 
specific explanations for how STEM CoPs and networks can be designed to create change and how their dynam¬ 
ics differ from those of the more organic networks that have been the focus of most social science research over the 
past fifty years. This report begins to answer the question of how STEM CoPs and networks can be best designed 
to maximize innovation. We explore what leadership and management is needed to support them, and how they can 
become sustainable. By understanding more about these features of communities of practice, we can design programs 
that better harness the power of such communities to bring about change. 

The project examined and compared four undergraduate STEM reform CoPs/networks that each have different 
designs, but share the common purpose of undergraduate STEM reform. The research sought to understand how 
such communities can be most effectively designed to spread innovations among network members, as well as on the 
campuses where those members are employed. 

A quick note about terminology. Social networks are defined in the literature as people loosely connected through 
some form of interdependencies, such as values, preferences, goals, or ideas (Wasserman & Faust, 1994). A communi¬ 
ty of practice (CoP) is a group of people who share a concern or a passion for something they do and learn how to do 
it as they interact regularly (Allee, 2000; Lave, 1988; Wenger, 1998 and 2007). A CoP has a greater sense of shared 
mission and purpose than a network, and it is often structured in a more intentional manner. 

While we entered the study considering that the four groups examined could operate either as networks, as commu¬ 
nities of practice, or as some hybrid of the two, by the end our research suggested that they are best understood as a 
particular variant of communities of practice, which we called “communities of transformation.” We will use this term 
to refer to the groups studied in this research, and in section 4 we will describe the important distinctions that define 
these communities. 


2 A few exceptions exist with the work of Mort at Columbia University in the 1950s. 
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This project addressed three main questions: 

1. How do members and leaders of communities of practice (CoPs ) perceive CoP design (membership, 
structure, communication, activities, and organization to support new knowledge development and 
action) shapes the ability to achieve goals (around undergraduate STEM pedagogical change and diffu¬ 
sion)? 

Sample sub-questions addressed: 

a) How can active engagement be obtained among members? 

b) What knowledge is best transmitted through networks, and through which media vehicles (on¬ 
line, in person workshops, etc.)? 

2. What are the perceived benefits of participation in a STEM reform community of practice or network for 
the individual participants and for their campuses? 

3. How do communities of practice and networks form, and how are they sustained in ways that help them 
to achieve their goals? 

Sample sub-questions addressed: 

(a) What leadership and management is needed to form and sustain these communities? 

(b) What barriers are there to formation and sustainability? 

(c) What strategies work best for overcoming barriers? 

* We conceptualized benefits throughout the research process in many ways, including benefits for individuals and institutions, 
outcomes of participation, and impact of these communities on individuals and institutions. As a result, we use the terms 
benefits, outcomes, and impacts interchangeably throughout this report. 


The Four Communities of Transformation 


What follows is a brief overview of the communities of transformation (CoTs) studied, and why they were 
chosen. A chart comparing these four communities is provided in the supplementary materials, and fur¬ 
ther details can be found on websites noted in the text. The four CoTs chosen for this project were Project 
Kaleidoscope (PKAL), the Process Oriented Guided Inquiry Learning Project (the POGIL Project), Science 
Education for New Civic Engagements and Responsibilities (SENCER), and the BioQUEST Curriculum 
Consortium. The study selected these particular groups rather than others in order to focus on communi¬ 
ties with the following key features: 

1. Focus on reform of undergraduate STEM education; 

2. Large-scale membership and wide dissemination of best practices; 

3. Higher education community and focus on reform within the context of postsecondary education; 

4. Long enough history to allow study not just of formation but also of outcomes and sustainability; 
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5. Ability to survey members 3 4 

These four communities met these criteria and also were different enough in key dimensions around design, 
organization, and activities to enable us to explore meaningful differences and to identify key patterns that 
can help future STEM networks as they form (or reform) to aid in STEM efforts. 

PKAL 

Project Kaleidoscope (PKAL) is an umbrella network of STEM faculty across the country that focuses on 
creating innovation among faculty to enable changes in their practices. The national PKAL community has 
nearly 7,000 members at over 1,000 colleges, universities, and organizations. Formed in 1989, PKAL was 
one of the earliest undergraduate science reform networks, and it continues to operate as one of the few 
networks to be sustained over time. More information about PKAL is available at http://www.aacu.org/ 
pkal . PKAL was chosen because it has several long-standing networks within it, and research that is limited 
to new or emergent networks is difficult and often does not yield as much data about long-term benefits or 
strategies for sustainability. Furthermore, PKAL includes several different types of networks, which lends 
itself well to comparative studies and to examining design differences and their effects on goals. This study 
focused on two ongoing networks within PKAL. The first is the Faculty for the 21st century (F21) network, 
which has been in existence for fifteen years. This is a loose network of nearly 1,500 STEM colleagues, of 
which nearly 200 have participated in PKAL summer leadership institutes. The second sub-community at 
the center of this study is the more recently formed family of regional networks that engage over 650 STEM 
faculty at 100 institutions. The regional networks develop and share effective models for transferring STEM 
education best practices and innovations among peers in order to increase the number of faculty members 
in each region who are using proven, research-based pedagogies. These regional networks are tighter and 
denser than the national network, and they have more regular interaction built into their structure. PKAL 
was chosen based on its long history, broad reach, advocating of several different pedagogical practices, and 
involvement of several different disciplines. 

THE POGIL PROJECT 

The Process Oriented Guided Inquiry Learning Project (the POGIL Project) is a national professional 
development and curriculum reform effort whose mission is to connect and support educators from all 
disciplines interested in implementing, improving, and studying student-centered pedagogies and learning 
environments. It involves approximately 6,500 faculty across a range of disciplines. The POGIL Project be¬ 
gan in 2003 with support from the National Science Foundation. It works to disseminate specially designed 
activities that express its instructional philosophy, and it provides professional development to faculty who 
are interested in implementing group-learning approaches and developing new instructional activities. It 
originated in the discipline of chemistry, but its approach has been disseminated into other STEM fields. 
While the network has long been based out of Franklin and Marshall College, the POGIL Project contin¬ 
ues to grow each year. The POGIL Project has recently become an independent 501(c)(3) as a strategy to 
sustain activity in the future. It has many subgroups and projects that have developed within the overarch¬ 
ing framework, and it boasts a strong set of regional networks. More information about the POGIL Project 


3 We also want to note why we did not choose other types of networks or communities. There are many networks that fo¬ 
cus on the link between K-12 and higher education (e.g. ISTEM) to improve teaching of STEM in high school and to ease 
transition into college. These partnerships, while important, involve a different sort of network that crosses different com¬ 
munities. These types of partnerships are not comparable to the networks in this study. Additionally, they are the one type of 
STEM reform community that has received some study and attention—less so as networks but as partnerships. There are also 
networks represented in disciplinary societies, but these focus mostly on scholarship, rather than teaching. Lastly, there are 
smaller communities of only a few dozen educators, such as the National Numeracy Network, but these groups are intimate 
and have limited reach. 
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is available at http://pogil.org/ . The POGIL Project was chosen for this research because of its stature as a 
long-standing network working toward a plan of sustainability in STEM reform. As an example of a com¬ 
munity of practice, it offers unique resources and varying forms of communication among its members. 

It was also selected because of its targeted focus on a particular pedagogy and a particular community of 
participant faculty, a strategy that differs from the broader-reaching approach exemplified by PKAL. 


SENCER 

Science Education for New Civic Engagements and Responsibilities (SENCER) is a faculty development and 
STEM education reform initiative launched in 2001 under the National Science Foundation's CCLI national 
dissemination track. SENCER is an approach to STEM education that teaches rough complex, capacious, 
contemporary, and contested civic challenges to basic canonical STEM knowledge and methods. It strives 
to use context to engage interest, to make science real and relevant, and to stimulate memorable learning. 
The project has expanded from focusing on single courses to smaller course modules, course intersections, 
learning communities, major curricular reforms, pre-medical and graduate education, new certificates, and 
degree granting programs. The SENCER community includes thousands of faculty members, academic 
leaders, and students from more than 400 two- and four-year colleges and universities in 46 states and nine 
countries. The organization’s goals are to: 

1. Get more students interested and engaged in learning in STEM courses; 

2. Help students connect STEM learning to their other studies; 

3. Strengthen students’understanding of science and their capacity for responsible work and citizenship 

SENCER was selected because it involves several different science disciplines like PKAL, has a broader 
range of pedagogical practices it advocates, and uses unique approaches to engage members such as team 
participation. 

KoQUEST 

The BioQUEST Curriculum Consortium has a 25-year history of supporting undergraduate biology 
education reform. It supports international and interdisciplinary collaborations among faculty, with the 
overarching goal of creating learning experiences that more accurately reflect biological science practices. 
The BioQUEST approach emphasizes student engagement in problem-posing, problem-solving, and peer 
persuasion. BioQUEST uses modern information and communications technologies as a means to increase 
student access to scientific data, tools, literature, and communities. Many BioQUEST projects involve part¬ 
nerships with scientific and educational organizations to provide professional development and innovative 
curriculum resources. The BioQUEST Curriculum Consortium includes a large network of faculty who 
have contributed materials and collaborated in the exploration of innovative biology education. BioQUEST 
was chosen because of its long history, reach to many members, advocating for a more targeted set of peda¬ 
gogical strategies, and focus in a particular discipline. 

We now turn in section 2 to the literature related to networks and communities of practice that framed and 
informed our study of these communities. 
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II. Framing: Social Networks and 
Communitiesof Practice 


W e drew primarily on two bodies of literature when designing this study: social network analysis 
and literature on communities of practice (CoPs). Here we review the literature that informed 
the study design, as well as pushed us forward in our analyses. 


Benefits of Networks 


Social network analysis focuses more on outcomes, as compared to the literature on communities of practice; thus 
we used social network analysis to frame our understanding of outcomes (described in section 5). Social networks are 
defined in the literature as people loosely connected through some form of interdependencies, such as values, prefer¬ 
ences, goals, or ideas (Wasserman & Faust, 1994). In general, outcomes are the most commonly studied aspect of 
networks. Since we had so much rich data in this area, this study focused on a narrower question: How might STEM 
networks be unique in terms of outcomes that promote change? 

Diffusion of innovation or change has emerged as an outcome in many different studies of social networks, which 
is why social network analysis has been applied to the study of change processes in more recent years (Rogers, 2003; 
Valente, 1995). Three specific outcomes of social networks have been related to change: learning, social capital, and 
risk-taking (Borgatti & Foster, 2003; Burt, 2000; Kilduff &Tsai, 2003; Tenkasi & Chesmore, 2003). Many research¬ 
ers have found a strong linkage between learning and changes in behavior, showing that as people interact with 
others in networks they are more likely to experience schema changes allowing openness to new approaches (Ten¬ 
kasi & Chesmore, 2003). Networks also provide social capital that facilitates the change process by providing access 
to relationships and knowledge about how to overcome barriers (Burt, 2000). While different definitions of social 
capital exist, most of the theoretical discussions operate under a definition of social capital as the resources embed¬ 
ded in social relations and social structure which can be mobilized by an actor to increase the likelihood of success in 
purposive action (Daly & Finnigan, 2009). These resources can vary to include knowledge about how organizations 
work, influence possessed by particular people, or access to financial resources. Finally, achieving long-term change 
often requires risk-taking that can be less problematic if it is done collectively rather than individually (Valente, 
1995). If a person knows that many of her peers are going to join her in an activity or behavior, she is more likely to 
feel empowered to engage in this behavior (Rogers, 1962; Valente, 1995). While there are other important compo¬ 
nents in the efficacy of networks, these three—learning, social capital, and risk-taking—are the ones most commonly 
identified that demonstrate the importance of social networks in achieving long-term transformations. There are of 
course many other individual and organizational outcomes of networks, but these three are the most often associated 
with change. 

As noted earlier, these outcomes identified in the literature have been drawn from studies of change initiatives 
in contexts that are quite different from that of STEM reform. Thus, as we examined learning, social capital, and 
risk-taking within this specific context, it was important to see which components were most salient in STEM 
professional networks, and whether other outcomes emerged as similarly important for change. As we studied the 
four STEM reform communities described in section 1, we paid close attention to how learning, social capital, 
and mediating risk-taking served to facilitate change in this particular environment. We also took into account the 
broader literature on outcomes 4 , and we drew on this literature as we studied those outcomes that emerged as pivotal 
to change within the networks. 
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Design of Networks 


While we knew much about outcomes of networks and benefits to members, we knew much less about how the de¬ 
sign of these networks shaped these various outcomes. This study set out to draw connections between design prin¬ 
ciples and the achievement of the outcomes described above that foster lasting change. The limited data that exist on 
this topic come mostly from the social networks literature, and we now turn to a review of that material. 

Various studies have identified how the design of social networks impacts outcomes such as achievement of goals 
and change. This literature, as well as the literature on design in communities of practice (described below) informed 
our data collection (section 3) and analyses related to design, described more fully in section 6. The most commonly 
identified design characteristics were: strong and weak ties, heterophily and homophily, subgroups, connectedness, 
and opinion leaders. We discuss each of these design attributes briefly below. 

STRONG TIES 

Strong ties are most useful for the communication of tacit, non-routine, and complex knowledge; in contrast, weak 
ties —present in networks said to be “less dense”—are better suited for communication of simple and routine infor¬ 
mation (Nelson, 1989; Tenkasi & Chesmore, 2003). Strong ties are characterized by three defining characteristics: 
frequent interaction, an extended history, and intimacy or mutual confiding between parties (Kraatz, 1998). Most 
studies of change find strong ties to be more conducive to deep or complex changes (Balkundi & Harrison, 2006; 
Tenkasi & Chesmore, 2003). Strong ties are more likely to promote in-depth, two-way communication and ex¬ 
change of detailed information. Because pedagogical and curricular reform efforts require a deep and complex kind 
of change, we anticipated that it was likely that strong ties would be important for undergraduate STEM reform 
networks. While they can be designed to create frequent interaction, networks that are created for the purpose of in¬ 
novation may be less likely to have extended histories or intimacy among members. Weak ties, on the other hand, are 
characterized by distance and infrequent relationships that may be casual, less intimate, and non-reciprocal in nature. 
However, for the dissemination of ideas and public information, weak ties can be extremely helpful. Weak links can 
also provide exposure to important external ideas that may promote the development of a more robust change idea. 
Thus, there may be times and circumstances where weak links are important for creating specific types of change, 
especially in certain phases of the change process. This study examined the degree to which networks can promote 
strong ties—those that have been found in the literature to be more useful for scaling up change. Also, the study 
examined when weak ties might be helpful in some aspects of network activity. 

HETEROPHILY AND HOMOPHILY 

Another area of design found to shape outcomes is diversity or homogeneity of ties. Diversity of ties, or heterophily, 
can lead to more complex thinking about change processes, but homogeneous ties, or homophily, can lead to quicker 
adoption of change and to stronger relationship development, ultimately encouraging strong ties (Borgatti & Fos¬ 
ter, 2003; Moody & White, 2003). Homophily might also lead to greater engagement and participation of network 
members. We can examine the degree of heterophily or homophily in the structure of social networks and draw 
conclusions on how those dimensions impact networks as they try to meet their goals. 

SUBGROUPS 

The development of subgroups (cliques) within networks has also been identified as a strong lever for moving changes 
forward (Freeman, 1979; Reagans &McEvily, 2003). Within the networks in our study, we examined the types of 
subgroups that form to facilitate change, and the structural properties that might govern this process. 


4 In terms of individual outcomes, the following have been identified: social support, sense of belonging, information sharing, 
community, more meaningful participation, enjoyment of work, confidence, help with challenges, expansion of skills, enhanced 
professional reputation, increased employability, and stronger sense of professional identity (Rogers, 2003; Valente, 1995). In 
terms of organizational outcomes, networks have been shown to help execute strategic plan, increase retention of talent, increase 
capacity for knowledge, allow for more alliances with external groups, foresee technological developments, improve quality of 
decisions, improve problem-solving, increase coordination across units, provide additional resources for implementing strategy, 
and strengthen quality assurance (Wenger, McDermott, & Snyder, 2002). 
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CONNECTEDNESS 

Another concept, connectedness , provides a measure of how much exposure each individual receives to the innovation 
(Borgatti & Foster, 2003; Valente, 1995). Individuals can be influenced to alter their behavior when they are sur¬ 
rounded by many people that have adopted a change, even if others throughout the campus or profession as a whole 
have not done so. This study tests the notion of connectedness by examining individuals who have more exposure to 
innovations than others within the networks. This approach yields insights on structural considerations for develop¬ 
ment of leadership and pervasiveness. 

OPINION LEADERS 

The presence of opinion leaders within social networks can help speed up adoption of innovations (Cross & Parker, 
2004; Freeman, 1979; Valente, 1995). We examined the way key opinion leaders from STEM are brought into net¬ 
works and helped to effect change. 

These five design features are discussed in the literature, but their treatment is typically constrained to the more or¬ 
ganic forms of social networks. While we examined each of these features in our study of STEM networks, we aimed 
to address a lack in the research in the question of how more constructed networks, like communities of practice, can 
be designed intentionally to achieve these outcomes. Thus, in addition to testing out some of the existing findings 
from social network theory about how design can shape outcomes, we explored new areas where little research exists, 
such as the type of leadership needed within networks, the way that formal organizations can support networks, and 
the way to create sustainable networks. 

Our approach expands upon the organic focus of social network analysis, which limits what we know about the role 
that can be played by leadership, organizational structures, and intentional support; this is a major gap in the litera¬ 
ture, noted by most social network scholars as a critical area for future studies (Mullen & Kochan, 2000; Spillane, 
Healey, & Kim, 2010). This gap is the reason there is so little literature about fostering and sustaining networks, a 
progression seen primarily as an organic series of events, rather than as a structured process. It is also important to 
note that social network analysis is primarily a survey-based, quantitative approach, which does not lend itself to the 
study of evolution of communities over time. Thus, we turned to the literature on communities of practice (CoPs) in 
order to partially remedy these limitations. CoPs have been studied through qualitative methods that better capture 
processes, and these studies have provided insight into the formation and sustainability of networks. There is also an 
existing body of research, albeit less well defined, about designing CoPs for engagement. 


Defining Characteristics of Communities of Practice 


A community of practice (CoP) is a group of people who share a concern or a passion for something they do and 
learn how to do it as they interact regularly (Allee, 2000; Lave, 1988; Wenger, 1998 and 2007). CoPs operate simi¬ 
larly to traditional social networks in that they connect people with a similar interest or value, and they similarly 
may cross organizational boundaries and be more loosely connected. CoPs are defined by three of characteristics: 1) 
a domain —a common ground of purpose and value; 2) a community —a set of individuals connected; and 3) a prac¬ 
tice —ideas, frameworks, tools, or documents that the community members share. Without these three areas, an entity 
cannot be defined as a community of practice. CoPs take on many forms: they can be co-located or distributed (i.e., 
centralized in the same location or not); short-term or long term; homogenous or heterogeneous (focused on the 
same field or area or more diverse); small or big; housed within a unit or organization or spread across multiple such 
units; spontaneous or intentional; unrecognized or institutionalized. While their forms can differ, CoPs often emerge 
within organizations, and they often involve people working day-to-day with one another. 

Because most research has focused on local CoPs within a single organization or unit, the principles identified in the 
literature on CoPs must be used with caution. Nonetheless, this research did provide direction for our methodologi¬ 
cal approaches, and it introduced some key insights—related to design, formation, and sustaining communities—that 
were valuable for shaping this study. In what follows, we review the defining characteristics of CoPs, as well as a 
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variant of CoPs known as professional learning communities (PLCs). These definitions helped us frame our under¬ 
standing of the communities in our study—communities of transformation—which are examined in more detail in 
section 4. What are these Entities? Communities of Transformation 

In the literature, CoPs tend to be organic, developing naturally from a need shared among people. Most of the early 
work on CoPs examined the tacit learning that happened while people worked together through apprenticeship 
practices. Lave (1988) and Wenger (1998), who originated the concept, philosophically disagree about whether CoPs 
can be intentionally created or structured and whether they are truly organic entities. Even Wenger, McDermott, & 
Snyder (2002), who went on to examine “non-organic” CoPs, feel that CoPs are defined by a more organic state, and 
that they can only be nurtured, not created. The community or social aspect of learning is central in this phenom¬ 
enon; thus, interaction and the relationships that are developed as part of the community are seen as essential to the 
CoPs existence. 

The key activity of a CoP is to develop the domain that is at the center of the community. This becomes the iden¬ 
tity of the community, and it serves as a focus for developing the shared repertoire of resources for the practice. The 
community operates through learning by problem-solving, sharing information, seeking expertise, visiting others, and 
using other similar approaches (Wenger et al., 2002). The literature describes challenges that emerge over the life- 
cycle or stages of a CoP. Given the organic nature of a CoP, it tends to go through a natural cycle (potential, coalesc¬ 
ing, maturing, stewardship, transformation) in response to challenges that result from growth. In the course of this 
cycle, a CoP refines its identity and membership, and it incorporates new members and new purposes (Wenger et al., 
2002). 

Professional learning communities (PLCs) are a particular type of structured and organizationally located CoP that 
is commonly found in the education sector. PLCs are built on the principle that community is central to learning. 
PLCs can be considered a type of CoP because each PLC involves a community , has a domain , and involves a set of 
practices (e.g., the socially defined practices that enable one to become a strong teacher or principal). PLCs are dis¬ 
tinctive, however, because they have several facets that are not part of the definition of CoPs. Lor example, PLCs are 
always intentionally created and tend to be heavily structured. This is less typical of CoPs. Additionally, the leader¬ 
ship of a PLC usually defines the membership, and a PLC typically includes people based on their roles, rather than 
on their organic interest in the domain (Stoll et al., 2006). While PLCs entail the exchange of information, expertise, 
and problem-solving, the character of these interactions in a PLC are often less peer-to-peer than in typical CoPs. 
While some institutions have set up peer-oriented PLCs with teachers only, these communities are always created 
and sanctioned by the institution’s administrative leadership (Bond & Lockee, 2014; Stoll et al., 2006). In addition, 
the nature of the work of PLCs tends to be more narrowly defined around a set of issues, such as student success or 
teaching broadly understood (DuLour, 2004; DuLour, DuLour, & Eaker, 2008). The model for such cases is more 
structured and hierarchical than that of CoPs (see, for example Roberts, 1998, in which PLCs are led by school prin¬ 
cipals). Domain, membership, and community operate differently in PLCs than they do in the other CoPs typically 
described in the literature. We now turn to the literature on design of CoPs. 


Designing Communitiesof Practice 


As with the limited literature on design of networks, the literature on design of CoPs informed our data collection 
and analyses, especially on the challenge of designing communities for engagement and the achievement of outcomes 
(see section 6). One of the major findings in the literature on CoPs is that design varies greatly based on the identi¬ 
fied goals. There are many different types of CoPs, so there is no single design that guarantees efficacy. While the 
social network literature emphasizes basic designs and structures that support certain outcomes, CoPs tend to exhibit 
a less direct connection between design and outcomes. There is not a single best design, but various design principles 
that can enable a community to meet its specific goals. Several such general principles and practices were identified 
in Wenger, McDermott, & Snyder (2002) as important for creating learning that leads to change: 

1. Design the community to evolve naturally. Because a CoP is dynamic by nature, in that its interests, goals, and 
membership are subject to change, it should be designed to support shifts in focus. 
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2. Create opportunities for open dialog not only among members, but also with those bringing in outside perspectives. 
While members and their knowledge are the most valuable resource of a CoP, it is also beneficial to look 
outside the community to understand other possibilities for achieving learning goals. 

3. Welcome and allow different levels of participation. Wenger identifies three main levels of participation in a 
CoP. First, there is the core group of members who participate intensely in the community through discus¬ 
sions and projects. This group typically takes on leadership roles in guiding the community. Second, there is 
the active group of members who attend and participate regularly, but not at the same level as the leaders. 
Third, there is the peripheral group of members who, while they are passive participants in the community, 
still learn from their involvement. Wenger notes that the third group typically includes the majority of the 
community. 

4. Develop both public and private community spaces. While CoPs typically operate in public spaces where all 
members share, discuss, and explore ideas, a CoP should also offer opportunity for private exchanges. A 
CoP designed in this way can coordinate relationships among members and access to resources through an 
individualized approach that is based on specific needs. 

5. Focus on the value of the community. A CoP should create opportunities for participants to explicitly discuss 
the value and productivity of their participation in the group. 

6. Combine familiarity and excitement. A CoP should offer the expected learning experiences as part of its 
structure, but there should also be opportunities for members to shape their learning experience together by 
brainstorming and by examining both the conventional and the radical wisdoms related to their topic. 

7. Find and nurture a regular rhythm for the community. A CoP should coordinate a thriving cycle of activities 
and events that allows for the members to regularly meet, reflect, and evolve. The rhythm, or pace, should 
maintain an anticipated level of engagement to sustain the vibrancy of the community, yet not be so rapid 
that it becomes unwieldy and overwhelming in its intensity. 


Lifecycle of Communities of Practice: Formation and Sustaining 


The literature on CoPs has developed a framework that examines formation, design, and sustaining of CoPs and is an 
expansion of the principles above, connecting them to how networks evolve over time. These various concepts were 
used to inform this study. The best-known framework for the lifecycle of CoPs was offered by Wenger et al. (2002), 
who created a five-stage community development model based on empirical studies of CoPs. We present this model 
here as a precursor to our description of the lifecycles of these communities in sections 7, 9, and 10. 

The lifecycle model includes the following stages: 1 .potential-, 2. coalescing. ; 3. maturing. ; 4. stewardship ; and 5. trans¬ 
formation (Wenger et al., 2002). Wenger and colleagues also outlined specific challenges or tensions for each stage, 
represented here in Table 2.1. These challenges indicate areas that might impact the viability or growth of CoPs. We 
briefly review the elements of the model that guide our exploration into higher education CoPs. 

POTENTIAL 

The first potential, is the phase where an important topic attracts an informal group of people who are inter¬ 
ested in beginning to work together. Wenger et al. (2002) note that at some point the “idea of forming a community 
is introduced into [a] loose network, and this prospect starts to redirect people’s attention. They start to see their own 
issues and interests as communal fodder and the relationships in a new light of a potential community” (p. 71). As 
the sense of this shared domain develops, more systemic planning and activities begin. So the beginning work at the 
potential stage is to define the scope of the domain that brings people together, to find people who see the value in 
increased networking and sharing of ideas, and to identify what common knowledge is needed to further the com¬ 
munity. Through this stage the emerging community creates a vision and sense of mission. 

COALESCING 

During the second stage, coalescing , people come together and launch the community, and they find value in engaging 
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TABLE 2.1: STAG ES OF COMMUNITY DEVELOPMENT AND KEY C HA LLENG EXTENSIONS FOR DEVELOPING COMMUNITIES OF 
PRACHC E (WENG ER ETAL, 2002) 


Stage 

C hallenge/Tension 

Potential: Community starts as a loose network of connec¬ 
tions with potential for growing and developing more con¬ 
nections 

Discover or Imagine: Build on what is present, or explore 
where potential could lead 

Coalescing: More connections are built, coalescing into a 
community 

Incubate or Deliver Immediate Value: Allow connections to 
form and build trust slowly, or immediately try to show the 
value of the community 

Maturing: Membership and depth of knowledge in the com¬ 
munity grows 

Focus or Expand: Direct energy toward internal interests of 
core members, or expand to meet interests of new members 

Stewardship: Actively share and develop knowledge formed 
through community 

Ownership or Openness: Balance ownership over community 
domain with the need to bring in new ideas 

Transformation: Community evolves as new members enter 
and/or initial energy wanes 

Let Go or Live On: Either let the community wane, or trans¬ 
form it in order to sustain the progress made toward goals 


in learning activities together. At this stage leaders in the CoP facilitate dialogue, create informal meetings, develop 
initial community support and communications, and develop organizational supports for the long run. Within this 
phase, the focus is on creating enough interest that people continue to participate. Part of creating this interest in 
continued involvement is achieved by establishing the value of the domain. The community needs to develop trust 
and strong relationships to get through philosophical challenges and other issues that emerge. The community also 
needs to develop key avenues for sharing information and creating information-rich resources. 

MATURING 

In stage three, maturing , the community begins to take charge of activities, and it grows in size. At this time, the 
community is involved in many joint activities together. Active learning is taking place, and the growing community 
develops standards for how its members interact over the long run. In the maturing stage, the community needs to 
clarify and focus its roles and boundaries. As the community grows, new ideas are brought in that might expand 
or change the domain of its focus. New members can disrupt the patterns of interaction that the core members of 
the community have developed. The community needs to find ways to stay focused on its core purpose and mission 
while it includes greater numbers of individuals. A key issue related to practice focuses on organizing resources and 
knowledge for the long haul; the community needs to systematize its practices and create a rhythm of activities that 
community members can count on. Also, the community identifies gaps in knowledge, especially as the community 
grows. One challenge is to keep creating additional resources to meet the needs of new members. The tension be¬ 
tween focus and expansion is palpable in this phase. 

STEWARDSHIP 

In stage four, stewardship , the community is well established and needs to find ways to sustain energy, to renew inter¬ 
est, and to continue to gain new members. At this point community leaders address organizational issues that may 
hinder their ongoing development, and they often forge linkages with other groups. In this stewardship phase, the 
community strives to sustain its momentum as continued new members join, as energies decline over time among 
longtime leaders, and as the original ideas of the community can fade in urgency and become less intellectually inter¬ 
esting. Stewardship is a balance between creating ongoing ways to bring in new ideas and remaining focused. Com¬ 
munities in this phase work to bring in new energy and new people, while supporting long-time leaders. Wenger et. 
al. (2002) describe the maturing and stewardship phases under the same broad label of maturing, and they see these 
two phases as hard to separate distinctly. 
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TRANSFORMATION 

The last stage is transformation. Wenger et al. (2002) note that a tension exists in this phase, between the commu¬ 
nity’s sense of ownership and its openness to new ideas—an openness that is never fully resolved and often results in 
crisis. As the community widens its boundaries, it risks diluting its focus. If the community stays closed, on the other 
hand, it can suffocate itself. It is a natural feature of the lifecycle of a CoP that these events should occur, and some¬ 
times the influx of new members in the transformation stage creates a new focus for the community; this leads it to 
transform. Other times the community may cease to exist, because members no longer feel that its purpose is relevant 
or needed. 


Localized and Distributed Communities of Practice 


Most of the literature reviewed to this point applies to more localized communities of practice. The STEM groups in 
this study, however, fall into what is called a “distributed community of practice” that cannot rely on regular face-to- 
face meetings and interactions as the primary vehicle for connecting members. Many communities of practice entail 
daily interactions among members, but distributed communities of practice have particular challenges that need to 
be considered in the design, formation and sustaining aspects: the distance that separates their members, the size of 
the community, and the need to work together across cultural differences (Wenger, McDermott, & Snyder, 2002). 
Because of the distance they cover, distributed communities must connect members more intentionally, and their 
design must think through challenges such as different time zones and the lack of spontaneous interactions among 
members. Second, distributed communities are often much larger in size than local communities of practice, with 
sometimes hundreds or thousands of members. Because people are unlikely to know each other well, or to have much 
face-to-face interaction, the community must wrestle with the question of the right size, and it must recognize when 
becoming too large impacts its viability. Thus, structures need to be created to adapt to growth. Third, distributed 
communities often run into issues related to the different cultures represented by their members. When people from 
across the country and world collaborate, they may not understand each other’s languages, customs, or styles of inter¬ 
action, and this can create communication barriers that eventually lead to problems within the community. This study 
acknowledges and takes into consideration these factors connected to distributed CoPs, which have not as yet been 
studied in depth. 


Summary 


This study was informed by the literature on networks and communities of practice. The network literature informed 
our original thinking about outcomes of community involvement and design, while the CoP literature informed our 
thinking about how these communities form and evolve, how they are sustained, and how they can and should be 
designed to maximize engagement. While both of these literature bases (social networks and communities of prac¬ 
tice) were informative, neither was a direct fit for the undergraduate STEM groups that were the focus of this study. 
These STEM reform groups are not organic, as is typically assumed in social network analysis, nor are they as tightly 
developed and structured as communities of practice. Instead, most STEM reform groups are semi-structured and 
fluid, and they best fit the model of a distributed community of practice—a model that has not yet been the object of 
much research. Therefore, a study of these unique STEM groups was needed to identify the outcomes, design, forma¬ 
tion, and sustainability issues that pertain to their work. We drew on the earlier research described above for con¬ 
cepts, theories, and framing, but, because we knew that the STEM groups in our study did not match the literature, 
we were open to new concepts and principles that emerged as we strove to understand their formation, design, and 
sustainability. We now turn in section 3 to the methodology we employed for this study. 
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II. Overview of Methods 


T he overall study employed an exploratory mixed-methods approach, including interviews, observa¬ 
tions, document analysis, and surveys. Studies of communities of practice (CoPs) typically utilize 
both quantitative and qualitative methods to identify trends and examine underlying mechanisms 
within the communities (Fontaine &Millen, 2004). In line with exploratory mixed-methods studies 
(Creswell & Plano Clark, 2011), we began with qualitative data collection of participant interviews of 
leaders and staff from each community, observations of signature community events, and document analyses for key 
documents from the four communities that we studied. This allowed us to better understand the design principles of 
and the nature of involvement in these communities as a means to explore this topic. We utilized findings from this 
phase of data collection in order to inform the survey design for the second phase. 


Data Collection 


OBSERVATIONS AND DOC UMENTANALYSIS 

The study began with a review of documents in order to develop a context for these four STEM reform communities. 
Samples of the types of items we collected include: notes from meetings, planning documents, advisory board cor¬ 
respondence, descriptions of their missions, philosophies, and values, key correspondence between leaders, grant ap¬ 
plications, reporting on grants, reports for advisory boards and other key groups, as well as on-going correspondence 
with the community via newsletters. Later, as part of interviews, we collected key documents that they identified that 
might help us to better understand what was engaging to them, such as publications, web-blogs, or newsletters. 

We observed a signature event for each community, visited each community’s main office (where we also went 
through their archives), joined their listservs, visited their websites on an on-going basis, and attended other key 
events about which they informed us. Observation took place over 2 and a half years. During observations, our re¬ 
searchers took fieldnotes about the activities, using the literature on communities of practice and professional learn¬ 
ing communities to develop observation protocols. Observation notes from key events were taken in each case by 
more than one researcher, and then they were compared for validity. Fieldnotes from events were typically quite long, 
30 to 35 single-spaced pages for each event. As Rogers (2003) has noted, we can learn a tremendous amount from 
real-time studies that follow networks and communities of practice, and from watching their activities. 

INTERVIEWS 

After initial review of documents and of data from site visits, we interviewed 112 people—between 26 and 30 people 
within each community (including both organization staff and faculty leaders). Each community studied is sup¬ 
ported by an organization that includes leaders and staff that have worked extensively with the communities, both in 
their current forms and in the past. The communities each have longstanding members and leaders that have helped 
sustain them; for this study, we drew on interviews with faculty leaders in particular. We also asked to speak with 
faculty who had less involvement in these communities in order to get a sense of their experiences as well. Interviews 
lasted between one and two hours, and they followed a common protocol that asked about impacts or outcomes from 
participating in the community, level of involvement, what they found most engaging in the community, what they 
perceived shaped the outcomes they noted, and other areas related to their engagement and involvement. The com¬ 
munities of practice literature informed the interview protocol. All interviews were recorded digitally and transcribed. 
Interviews were used to inform items for the survey and to build on the literature we brought to the study. 

For the interview portion of the study, our sample (n=112) consisted of 75% current faculty members (n=84)—60.7% 
of whom were professors (n=51) and 29.8% associate professors (n=25). The remaining 25% of participants (n=28) 
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were either former faculty members, current administrators, or staff members of the four reform communities. When 
asked to indicate their primary job responsibilities, 42% of the sample indicated teaching (n=47), 33.9% indicated ad¬ 
ministration (n=38), 3.6% indicated research (n=4), and 20.5% indicated other responsibilities (n=23). As for personal 
demographics, 57% of participants identified as female (n=64) and 92% identified as White (n=103). 

SURVEY 

The survey was conducted last, and the survey design was informed by the interviews, documents, and observations. 
The survey invitation was sent to 17,868 e-mail addresses. 5 The survey was custom designed for each community’s 
particular structures (e.g., activities, communication vehicles), but it followed a common survey design to allow for 
comparison across the four communities. It addressed the following areas: participants’involvement in the com¬ 
munity over time; perceptions of community activities; perceived outcomes of community involvement for individu¬ 
als, their departments, and their institutions; perceptions of the importance of community design elements on their 
participants’ practice; and individual and professional characteristics. Survey design was informed by the information 
gathered in the first phase of data collection, as well as by the literature pertaining to design and outcomes of net¬ 
works and communities of practice. This allowed us to identify the design aspects and involvement opportunities that 
characterized these communities. 

A total of 3,927 participants responded to the survey invitation, indicating a 22% initial response rate. This response 
rate is similar to the response rates of other surveys administered to national samples of STEM faculty (e.g., Hurta¬ 
do, Eagan, Pryor, Whang, &Tran, 2012). The final sample for this study consists of 2,503 participants who complet¬ 
ed the entire survey; these participants were distributed amongst 997 institutions (ranging from 1 to 28 observations 
per institution) and four communities (ranging from 235 to 1,102 observations per community). The survey sample 
consisted of 36.7% professors (n=919), 27.9% associate professors (n=699), 9.2% assistant professors (n=231), 20.2% 
non-tenure-track faculty or faculty working in institutions without tenure (n=506), and 5.9% individuals with no 
academic rank (n=148).The mean length of time spent teaching undergraduate students was 16.8 years (SD = 8.67). 
More than half of the participants worked in public institutions (n=1320,52.7%), and 21.2% worked in doctoral 
institutions (n=530), 32.6% in master’s institutions (n=816), 27.8% in baccalaureate institutions (n=695), 13.7% in as¬ 
sociates institutions (n=342), with the remaining portion working in other organizations or types of higher education 
institutions (n=120; 4.8%). As for personal demographics of the survey sample, 54.3% identified as female (n=1359), 
82.4% as White (n=2062), and the average age of participants was 49.9 years (SD = 10.5). 


Data Analysis 


The qualitative data were coded and analyzed using Boyatzis’ (1998) thematic approach. This approach involved first 
going through the data for new or emerging inductive codes. Second, deductive codes derived from the literature on 
communities of practice and learning communities was then applied. Deductive codes included items reviewed in the 
literature related to stages of CoP development and design principles, as well as items from the literature on learning 
communities. The qualitative data were analyzed using HyperRESEARCH, a qualitative software program that helps 
manage and analyze large amounts of qualitative data and eases the coding process. All forms of qualitative data 
including interviews, observation fieldnotes, and documents were inputted into the software. 

We utilized several quantitative analytical procedures to analyze quantitative data. Scale scores for our outcome vari¬ 
ables were calculated by averaging the individual items in each scale, rather than by summing the items, in order to 


5 The administrative staff of the four STEM reform communities provided us with contact information for each individual 
on their e-mail lists in order to send personalized invitations and to track responses. All four organizations acknowledged the 
existence of out-of-date contact information for participants and individuals who do not identify as faculty (i.e., members of 
other organizations) on their contact lists. Additionally, one community has a high-school arm of their initiative, and they were 
unable to separate those addresses from the larger list. So, while the population in the study was approximately 18,000, there is 
no way for us to know the true population size. 
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contribute to the ease of interpretation and comparison with other outcome items (Furr, 2011). We utilized descrip¬ 
tive statistics of our outcomes and design variables to identify trends in the data. 

We then utilized ordinary least squares (OLS) regression to examine the extent to which participants’ perceptions of 
CoP design characteristics and engagement are associated with the three dependent variables in our study. Prior to 
utilizing OLS regression, we examined the unconditional intraclass correlations (ICCs) for three individual outcome 
variables (learning and improving practice, skills for leadership and change, and networking) and two organizational 
outcome variables (departmental change and institutional change) because our participants exhibited clustering by 
institution and reform community. We opted to utilize OLS regression rather than multi-level modeling for two 
reasons. First, the majority of the variance in our dependent variables was within institutions rather than between 
institutions or communities. Second, our sample contained a large proportion of singletons in institutions, as well as 
institutions with only two participants (35.2%), threatening the estimates and validity of utilizing multi-level model¬ 
ing with these data (Rabe-Hesketh & Skrondal, 2012). 

Prior to running the regression models, we calculated descriptive statistics and examined histograms for each con¬ 
tinuous variable in the study to ensure approximately normal distributions. We also calculated multicollinearity 
statistics for all variables. Variance inflation factor (VIF) values were low (ranging from 1.04 to 3.42) and well within 
the acceptable range to indicate no issues with multicollinearity in the analyses (Meyers, Gams, & Guarino, 2006). 
We ran regression models that included focal variables (design characteristics and engagement behavior) and control 
variables (personal demographics, professional characteristics and motivations, and institutional characteristics). All 
continuous variables (including the dependent variables) were standardized (i.e., grand-mean centered) prior to their 
inclusion in the models. 


Trustworthiness and Validity 


We utilized multiple forms of trustworthiness, including outside experts and auditors, member checks, triangulation, 
piloting, and multiple coders. We had two advisory boards that informed the study design and reviewed results: an 
external board comprised of national STEM experts and an internal board comprised of members from each of the 
four initiatives studied. We presented data collection protocols and instruments, as well as findings, to each board for 
input. The internal board was able to serve as a member check and to register whether the findings seemed to reflect 
their insights and experience. We piloted the interview and observation protocols. We triangulated data from mul¬ 
tiple sources—documents, observations, and interviews. For the focus on sustainability, the key data were examining 
alignment or any discrepancies between interviews and archival data about development and sustainability. Lastly, we 
had three different coders of data that compared their interpretation of the emerging trends and coding of deductive 
codes within HyperRESEARCH. Coding was conducted separately and then compared. 


Summary 


The exploratory mixed-methods nature of this study addressed a set of research questions that spanned outcomes, en¬ 
gagement to lifecycle of the CoPs. The qualitative work allowed us to understand the ways in which the communities 
operate or engage faculty, how they formed, and how they have been sustained over time. This information contrib¬ 
uted to our ability to design a survey to community members in order to best understand the outcomes of participat¬ 
ing in these communities, and to identify how engagement (often in terms of design principles) in these communi¬ 
ties contributes to individual and broader outcomes in members’ departments and campuses. We now turn to the key 
findings from our study, beginning first with the finding that these communities can be identified and understood as 
a variant of communities of practice. This model, which we call a “community of transformation,” encapsulates how 
we think these communities work scaling STEM reform. 
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IV. What a re These Entities? 

C o m m u n itie s o f Tra n sfo mn a tio n 


A s we described in section 2, we originally framed the study using the literatures on social networks 
and communities of practice (CoPs). As we studied these four groups, however, it became clear that 
they did not share many of the characteristics of social networks. They had more structure than most 
networks, they were more formal, they had stronger shared purposes and goals, and their members 
had closer ties. We did find that individuals in these communities formed their own informal net¬ 
works of connections through their involvement. However, we also found that the ways in which members engaged 
with their communities were heavily influenced by the ways in which the communities themselves were fostered by 
the leadership and by shared philosophies. Thus, community members participated according to organizational struc¬ 
tures, rather than according to the informal relationships that were developed through involvement. In other words, 
network concepts failed to explain the workings of these groups. 


STEM Reform Communities in Relation to the Communities of Practice 
Literature 


In comparison to the literature on networks, the community of practice literature was much more relevant for under¬ 
standing these STEM reform communities’characteristics. The groups each had a clear domain—a type of teaching 
innovation—on which they were focused. This domain created a strong identity for members, which was common in 
CoPs. There was also a sense of care for domain and work that is common of CoPs—not just a sense of shared inter¬ 
est that characterizes social networks. The communities were well formed and nurtured, additional attributes that are 
characteristic of CoPs. The sense of community served as a strong social fabric in each group, and it was essential to 
fostering the domain. Further, each community focused on a practice: teaching and developing resources and shar¬ 
ing relevant information. In our interviews, it became clear that participants in these groups described themselves as 
members of communities, and that the language of networks was foreign and did not resonate with their experiences. 

The literature on CoPs was very helpful for explaining the formation of these groups and their common lifecycles 
and challenges. The basic design principles from the literature on CoPs were relevant for engaging members in these 
groups, although, as we highlight in this report, the most important design aspects we found at work to facilitate 
engagement were not reflected in the CoP literature. The outcomes of these groups were also similar to those found 
for other CoPs related to learning, leadership, networking, and re-energizing people. Thus, we found much resonance 
between the findings of our research and the existing literature of CoPs; the literature can be a helpful base to draw 
on to inform future efforts at scaling STEM reform among faculty communities. 

However, the literature on CoPs did not fully describe the phenomena that we recorded in our research. For example, 
while communities are structured in many ways, most empirical studies tend to document CoPs that are located 
within organizations. CoPs in the literature thus tend to exist within a company, hospital, or government agency and 
be supported with space, resources, materials, staffing, and leadership through the sponsoring organization, whether 
formal or informal. Each of the STEM reform communities we studied was not situated in this way, which led us to 
describe them as non-organizationally located communities. By non-organizationally located, we mean that there is not 
an organization providing resources (human and financial) or other infrastructure to the communities. This position 
presented specific challenges to the communities studied—particularly challenges around sustainability. Perhaps as a 
result of this difference, the communities in this study had a divergent approach to issues of expansion, as compared 
to that described in the CoP literature. Therefore, we found their non-organizational status to be an important dis- 
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tinction for the entities we were studying—a distinction that significantly influenced results, as we examine in more 
detail in the ensuing sections. 

Additionally, in the entities studied, we also found some meaningful differences related to engagement and design 
that were not captured in the CoP literature. As a result of these differences, which we describe in detail below, we 
labeled the groups we studied a variant or subtype of communities of practice, called communities of transformation 
(CoTs). In this section, we describe how the CoTs in this study resemble CoPs, but they also bring some unique fea¬ 
tures that made our results different from the more general CoP literature. A comparison of CoTs with other com¬ 
munities we described in section 2—CoPs and professional learning communities (PLCs)—can be found in Table 
4.1. 


Communities of Tra nsformation 


In this study, we believe that we have identified empirical 
data to support another variant of communities of practice 
called communities of transformation, or CoTs. To best un¬ 
derstand CoTs, it is helpful to compare and contrast them 
with CoPs. 

Traditional CoPs tend to work within the value system of 
their organizational settings to improve those settings. The 
practices that they put forward are not seen as entailing a 
dramatic departure from the status quo; rather, they offer 
improvements on existing efforts that can be understood 
within the philosophy or paradigm of existing practices. 
What we found in our interviews and through observing 
events and activities is that a CoT departs significantly 
from existing practices and values to create an innovative 
culture and reality. When people participate in these com¬ 
munities, they are introduced to and over time begin to live 
new practices that dramatically depart from the practices 
currently used within their institutions. Traditional CoPs 
also often operate in more organic, gradual ways, as people 
learn from each other through day-to-day practice. In the 
distributed communities that were the focus of this study, 
however, participants did not have such daily interactions 
to drive their learning, and they only experienced intermit¬ 
tent contact with the broader communities. In this setting, 
specific learning mechanisms were established by these 
communities that engaged individuals across isolated loca¬ 
tions. This was an important design feature for the success 
of these CoTs. 

We found three defining elements crucial for creating new 
or innovative cultures in CoTs, distinctive from CoPs and 
PLCs: 

1. A compelling philosophy; 

2. Living integration of the philosophy throughout 
activities and communications, creating a new 


APPENDIX 4A: Sample of Philosophy in a Guid¬ 
ing Document 

Directly quoted from SENC ER website (SENCER, 2015): 

The SENCER Idea Is illustrate the principlesand phi¬ 
losophies that guide SENCER'sapproach to educa¬ 
tional practice: 

• SENCER robustly connectsscience and 
civic engagement by teaching "through" 
complex, contested, capacious, current, and 
unresolved public issues "to" basic science. 

• SENCER invitesstudentsto put scientific 
knowledge and the scientific method to 
immediate use on mattersof immediate 
interest to students. 

• SENCER helps reveal the limits of science 
by identifying the elements of public issues 
where science doesnotoffera clearresolu- 
tion. 

• SENCER shows the powerof science by iden¬ 
tifying the dimensionsof a public issue that 
can be betterunderstood with certain math¬ 
ematical and scientific waysof knowing. 

• SENCERconceivesthe intellectual project 
aspractical and engaged from the start, as 
opposed to science education models that 
view the mind asa kind of "storage shed" 
where abstract knowledge may be secreted 
forvague potential uses. 

• SENCERseeksto extractfrom the immedi¬ 
ate issues the larger, common lessonsabout 
scientific processesand methods. 

• SENCER locates the responsibilities (the bur- 
densand the pleasures) of discovery asthe 
work of the student. 

• SENCER, by foe using on contested issues, 
encouragesstudent engagement with "mul¬ 
tidisciplinary trouble" and with civic ques¬ 
tions that require attention now. By doing so, 
SENCERhopesto help studentsovercome 
both unfounded fearsand unquestioning 
awe of science. 
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world of practice; 

3. A network of peers to break the isolation, brainstorm revising practices, and help sustain changes once an 
individual returns to the status quo environment 

The overall elements of CoTs are mapped onto a chart (Table 4.1) that also compares them to CoPs and PLCs. We 
will refer to this chart more specifically in the discussion, and it also captures elements presented in the findings. 

PHILOSOPHY 

First, having an engaging, well-articulated, and clear philosophy is important to ground people in a new value system 
and to guide novel behavior, especially when educators are in isolation, trying to learn and practice alone at their in¬ 
stitutions. This philosophy provides an anchor for learning. In the interviews, faculty discussed how the philosophies 
of these four communities were the most compelling and engaging aspect of their involvement, and the survey results 
of these four organizations also reinforce this point (Kezar & Gehrke, 2015a). An example of one of these guiding 
philosophies is provided in Appendix 4A: the SENCER Ideals. Importantly, the underlying philosophies of these four 
CoTs challenge traditional ideas of science, not only teaching practices. For example, SENCER’s focus on relevant 
social problems, civic education, and interdisciplinarity is a departure from more traditional, disciplinary views of 
science. Similarly, BioQUEST has brought philosophers into its community and encourages creativity and interdisci¬ 
plinary thinking, and PKAL examines the cultural and social underpinnings of science and the importance of being 
culturally relevant and student centered. By challenging traditional notions of science, these groups advance entirely 
novel approaches, not just the limited practice of tweaking science curriculum. 


Three defining elements of communities 
of transformation are: 1. A compelling 
philosophy; 2. Living integration of the 
philosophy throughout activities and 
communications, creating a new world 
of practice; and 3. A network of peers to 
break the isolation, brainstorm revising 
practices, and help sustain changes once 
an individual returns to the status quo 
environment. 


LIVING INTEGRATION OF PHILOSOPHY 

Second, in order to help members to embrace new practices that 
depart from the status quo within their own institutions, it is 
critical that communities provide members with opportunities 
to live or embody their values in practice. CoTs help individuals 
to inhabit new possibilities by creating this novel, philosophi¬ 
cally driven world of practice. Thus, we found it to be another 
core characteristic of CoTs that their philosophies be embodied 
through a number of activities and events. For a CoT to embody 
its philosophy means that its signature events operate accord¬ 
ing to the philosophy, that the leadership of the community 
exemplifies the philosophy in all communications (e.g., listserv, 
websites), and that materials (e.g., resources, texts) reflect the 
philosophy. As community members articulated the influence of 
these different areas—activities, leadership, and resources—they 


noted how each area distinctly contributed to the eventual transformation that the community made possible. 


NETWORK OF PEERS 

Third, the relationships faculty formed in the communities helped them to maintain their new practices when they 
returned to their campuses, in part by allowing them to brainstorm uses for the practices on their own campuses. 

This feature of these CoTs is similar to common designs of CoPs in general, and of PLCs in particular, but it also 
introduces a different approach to mentorship and support. All communities of practice are based on the premise 
that interpersonal support for change and innovation in practice is important; thus, it is typical to see the importance 
of community and relationships within any derivative of a CoP. Yet, in traditional CoPs, relationships are often tacit, 


6 Parallel to this effective structure of mentorship and community building, participants also discussed regional networks 
that were created by each of these CoTs. Participants said that such sub-networks typically did not represent the CoT well, 
because the regional communities lacked the key leaders to embody the philosophy, did not have an infrastructure where the 
philosophy was built into the design, and did not have enough of a cadre of dedicated volunteers. While all those interviewed 
recognized that the regional networks could perhaps one day themselves become communities of transformation, at pres¬ 
ent they fell short of what the more central efforts were able to create. It is these elements that proved crucial to the success 
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occurring naturally as people work with one another. In contrast, within PLCs, considered in this study to be a subset 
of CoPs, community development and learning are highly structured; it is the role of learning community leaders to 
arrange regular sessions for group brainstorming and sharing of information. Within the CoTs studied, however, this 
process has unfolded in a way unique from these two alternatives. Each CoT has developed a core of individual vol¬ 
unteers that provide mentorship and ongoing communication with individuals who attend events. At all times, these 
four communities have a group of individuals—usually 100 to 150 people—that are willing to communicate with and 
mentor a set of members as they begin to live the new practice. Usually these relationships develop organically based 
on shared or similar disciplines, institutional types, backgrounds, or concerns. At other times, these pairings can be 
assigned, such as when an individual member appears not to be making organic connections. These relationships are 
not highly structured as in PLCs, nor are they tacit as in CoPs—rather, relationships are more loosely connected and 
not formally dictated by overall structures. The faculty that are mentored then in turn become mentors for the next 
group. Each STEM reform CoT used this type of multi-generational approach, ensuring that new professionals are 
constantly recruited, that a middle generation is continually being mentored into more advanced roles, and that a se¬ 
nior group reliably works in leadership roles. Because these CoTs have been around for many years, they have reiter¬ 
ated this core group of individuals many different times, and they are constantly rejuvenating each group. We identify 
not only the importance of mentorship and relationships related to transformation, but the vehicles that emerged for 
developing and sustaining these volunteers over time. 6 

CoTs are similar to traditional CoPs in many aspects: they are organic, they share the underlying characteristics 
identified in the literature (domain, community, and practice), and they nurture membership and a community that 
mirrors that of CoPs. Yet, communities of transformation are distinctive in several key characteristics: 

1. They focus on creating and fostering an innovative space that does not exist; 

2. They rely on philosophy more than practice as they work to define the domain; 

3. The philosophy is central to their community adhesion, engagement, and action 

See Table 4.1 for a summary of the qualities of CoTs and a comparison with the characteristics CoPs and PLCs. 
While communities of practice are typically focused on improving specific practices (e.g., improving customer service 
or dental hygiene), they are not often engaged in radically rethinking or altering that practice. In contrast, innova¬ 
tion across theory and practice—not simply modification of practice alone—is a defining feature of communities of 
transformation. A philosophy is the coalescing feature that serves to embed innovation. Within CoTs, the domain 
is more than an interest area; it is distinctly defined by a philosophy and underlying values. The community supports 
this philosophy by living it through various interactions and through the work that is carried out both internally in 
the community and externally on members’ campuses. Events, communications, and all community-related activities 
are defined by the task of living this philosophy. We believe that philosophy played such an important role to these 
communities because the practices they espoused were innovative and challenged the existing status quo. Thus, the 
practical innovation required a clear rationale and articulation. 

In terms of community, CoTs have similar activities to traditional CoPs. Each shares information, establishes men¬ 
tors, seeks and fosters expertise, and solves problems. However, we saw a unique quality of CoTs in the way these 
community activities were all defined by enacting the philosophy to engage the community. This was reflected in the 
descriptions given by community members that described the philosophy as the most salient feature. Also, the com¬ 
munity relationships were neither tacit/organic (as in traditional CoPs) nor highly structured (as in PLCs). Instead, 
they were intentionally designed using organic elements. This design included the emergence of leaders through 
several avenues, as well as the creation of structures such as key annual events, communication channels, websites, and 
ways to link various faculty together to brainstorm and mentor one another. 


of the core communities that we capture in the narratives below. Note that PKAL was an exception to this finding, and its 
regional networks were stronger and more successful compared to those of the other communities studied. 
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Table 4.1: Comparison of Core Characierisiics of Communihes of Practice, Professional Learning Communities, 

AND COMMUNmES OF TRANSFORMATION 


Characteristic 

Community of Ractice 

Professional Learning 
Community 

Community ofTransformation 

Definition 

Group of individuals who share 
a concern or a passion for 
something they do and learn 
how to do it better as they 
interact regularly. 

Group of individuals commit¬ 
ted to working collaboratively 
in ongoing processes of collec¬ 
tive inquiry and action research 
to achieve better results. 

Distributed community of individuals 
that uses a core philosophy to create and 
foster new practices that can be integrat¬ 
ed into the various institutions in which 
individuals work. 

Underlying 

characteristics 

A domain, a community and 
a practice that is shared across 
participants. 

A well-defined domain, a hier¬ 
archical and structured com¬ 
munity, and often not a clear, 
shared practice. 

An innovation that is lived (domain), a 
distributed community, and a practice 
(e.g., teaching STEM). 

Membership and 
domain 

Identity is defined by a shared 
domain of interest in current 
practices. Membership implies 
a commitment to the domain, 
and a shared competence that 
distinguishes members from 
others. Members are practi¬ 
tioners who develop a shared 
repertoire of resources: experi¬ 
ences, stories, tools, ways of 
addressing recurring problems, 
etc. 

Membership is defined often 
by a leader who created the 
community; thus, the identity 
of the PLC comes jointly from 
the domain as well as from the 
leader. In education PLCs, the 
domain is typically student 
success. The notion of a shared 
practice may not be a prevalent 
part of this model. 

Shared interest or domain is an inno¬ 
vation that does not currently exist in 
practice in a substantial way; members 
are organized around the task of bringing 
this vision into practice. Membership is 
organic, as in CoPs, and there is a shared 
practice (i.e.. teaching). 

Community 

Members engage in joint 
activities and discussions, help 
each other, and share informa¬ 
tion. They build relationships 
that enable them to learn from 
one another. The focus is on 
improvement of the domain. 
Traditionally, CoPs have been 
physically located in one place 
and have expanded over time. 

Membership is steered toward 
the explicit task of bringing 
together teachers and adminis¬ 
trators, or other hierarchically 
defined practitioners. Across 
this hierarchy, a sense of collec¬ 
tive work is emphasized, such 
as efforts toward renewal or 
improvement of a school. 

Members engage in joint activities and 
helpful discussions mostly shared at a 
distance. Their relationships enable them 
to learn or share from each other. The 
focus is on engagement and absorption 
of a novel practice. Communities rely on 
a hybrid structure with some in-person 
encounters, relying mostly on distance 
interactions. These communities are less 
organic than CoPs and less structured 
than PLCs. 

Actions 

Problem-solve, share informa¬ 
tion, seek and foster expertise, 
visit others, map knowledge. 

Discuss teacher work, discuss 
student work, discuss student 
data, discuss the professional 
literature. 

Hold signature events that demonstrate 
the new innovations; develop leadership 
that embodies this new goal; develop a 
guiding philosophy that helps support 
the new practices; create a guiding docu¬ 
ment. 

Research back¬ 
ground 

Lave and Wenger s concept of 
situated learning, developed 
while studying apprenticeship 
as a learning mode. 

Evolution of Lave and Wenger 
into a highly structured, con¬ 
structed, and hierarchical form 
of situated learning. 

Further evolution of Lave and Wenger, 
not situated in day-to-day practice, but in 
a distributed community. Development 
of idea of community that is neither fully 
organic nor highly constructed. 

Where applied 

CoPs have been adopted most 
readily in business due to the 
recognition that knowledge 
is a critical asset that needs to 
be managed strategically. Also 
seen across multiple sectors 
(government, non-profit) and 
professions like academe and 
law. 

PLCs are mostly used in 
schools and in other more 
hierarchical institutions. Also 
found in other professions. 

To date, CoTs have only been identified 
in higher education, but they are likely to 
exist in other places. They are most likely 
to be useful in settings or domains where 
a deep or fundamental change in practice 
is needed or already taking place. 
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It is also striking that the four CoTs we studied were distributed (located in a network, not in a single organiza¬ 
tion) and hybrid communities. By hybrid communities, we mean that they employed a combination of virtual and 
in-person connections; we describe this feature under future research, as we did not have evidence that it constitutes 
an essential quality. The distributed nature of these CoTs, however, seems to be an essential characteristic of such 
communities; CoTs function through the power of a distributed community to support people in isolated status quo 
locations. 

The findings regarding distributed structure of communities of transformation should be treated with caution, how¬ 
ever. While we describe this characteristic as a unique aspect of CoTs, it may also be a result of the fact that tradi¬ 
tional communities of practice have historically been studied only within organizational settings; thus, much of the 
empirical literature reflects this form. CoPs typically have been seen to be organizationally situated and working to 
modify existing practices, but largely not challenging the existing order of the organizations that support them. Thus, 
we would like to acknowledge that the broad definition of CoPs indeed allows for the possibility of more radically 
altering practice, even if this function has so far not been documented in empirical research. By elaborating on the 
radical innovations fostered by communities of transformation, we hope to bring to light a variant empirical example 
of communities of practice that has not been identified in the literature to date and to articulate and define its key 
characteristics. 

While we identified communities of transformation within higher education settings, we imagine they may be 
common across many sectors, both within non-organizational settings and among more networked groups. We also 
imagine that philosophy may be more relevant to certain types of practices, such as teaching, that are particularly im¬ 
bued with complex beliefs that define them. For example, medical practitioners might form a community of transfor¬ 
mation to introduce a radical variation into their complex practice, such as embedding acupuncture into traditional 
medical practice, which would entail a dramatic departure from status quo practice in many institutions. A group that 
formed around this issue may also be defined as a CoT. In light of this expansive applicability, we imagine that there 
are other CoTs that exist in various fields that have simply not been identified to date. Our research can be instruc¬ 
tive to such CoTs, whereas the existing research on CoPs may be misaligned for outlining the best ways to engage 
participants, for understanding lifecycles and challenges faced, or for best understanding the outcomes possible for 
such groups. 


Summary 


In summary, through this study we have identified a new variant of communities of practice, which we have termed 
communities of transformation (CoT). These communities have three distinguishing characteristics that differentiate 
them from CoPs. They exhibit: 

1. A compelling philosophy; 

2. Living integration of the philosophy to create a new world of practice; 

3. A network of peers to break the isolation and brainstorm revising practices 

This approach to STEM reform holds promise in the ways that it can engage faculty and contribute to relevant ben¬ 
efits and outcomes for STEM education. In section 5, we describe these outcomes and benefits. 
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V. Outcomes and Benefits from 
Pa rticipation 


O ne of the goals of our research was to identify outcomes and benefits of participation in communi¬ 
ties of transformation (CoTs) as perceived by community members. This was in line with our second 
research question: What are the perceived benefits of participation in a STEM reform community 
of practice or network for the individual participants and for their campuses? In this section, we 
describe the outcomes we identified through our research, as well as some trends in the data regard¬ 
ing these outcomes. Later, in section 6, we look in more depth at the ways that engagement and community design 
are associated with these outcomes. 


We asked participants to assess the extent to which their involvement in their communities contributed to meeting 
26 individual-oriented benefits and 13 outcomes related to their organizations. Based on the literature on communi¬ 
ties of practice (CoPs), we had an idea of the kinds of outcomes that might be met through involvement in the four 
communities of transformation. The outcomes we settled on were informed by this literature, as well as by qualitative 
data gathered through interviews and observations. As you can see in Tables 5.1 and 5.2, we performed exploratory 
factor analysis on these items and uncovered five broader constructs on which 30 items could be grouped. Three of 
these outcomes relate to the individual, including learning and improving teaching practice, skills for leadership and 
change, and networking, while the other two pertain to departmental and institutional outcomes. The remaining 
items that did not load on a construct represent other important outcomes that contribute to STEM reform (see 
Table 5.3). 7 


Table 5.1: Scale Scoiesand Factor Loadings for Individual Outcomes Scales 


Individual Outcome - Scales a 

Factor Loading 

Cronbach 

a 

M (SD) b 

Learning and Improving Practice 


0.95 

3.33 (1.05) 

-Led to changes in teaching practice 

0.90 



-Motivated me to be innovative in practice 

0.89 

-Led to professional growth to improve practice 

0.82 

-Gained access to new curricular/pedagogical resources 

0.79 

-Contributed to intellectual growth 

0.77 

-Provided examples to model work after 

0.68 

-Contributed to understanding big picture of STEM reform 

0.57 


NOTE: a Factor loadings and Cronbach as calculated for total sample only, b Five-point scale with 1 =Not at all, 3=To some extent. 


and 5=To a great extent. 


7 Tables 5.1 and 5.2 list the factor loadings and Cronbach a values for each scale. The factor loadings for each item represent 
the extent to which each item is correlated with the underlying factor, with a value of 1 being the strongest possible relation¬ 
ship to the factor. For example, the item most strongly correlated with our first individual outcome Learning and Improving 
Practice is the item “Led to changes in teaching practice.” The Cronbach a value represents the internal reliability of the items 
within the scales, again with 1 representing the strongest internal consistency of each scale. 
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Table 5.1: Scale Scoresand Factor Loadings for Individual Outcomes Scales 


Individual Outcome - Scales a 

Factor Loading 

Cronbach 

a 

M (SD) b 

Leadership/Change 


0.94 

2.63 (1.13) 

d-Gained skills to overcome barriers to change 

0.85 



-Empowered to influence change on campus 

0.76 



-Helped develop skills to be a leader 

0.72 



-Motivated me to overcome barriers at home institution 

0.72 



-Gained ideas for contributing to change on campus 

0.60 



Networking 


0.91 

2.54(1.11) 

-Expanded personal support network 

0.87 



-Connected to people who share personal interests 

0.86 



-Connected to people who share professional interests 

0.77 



-Expanded professional support network 

0.73 



-Connected to a local (i.e., geographic proximity) network 

0.67 



NOTE: a Factor loadings and Cronbach a 5 calculated for total sample only, b Five-point scale with 1 = 
and 5=To a great extent. 

Not at all, 3=To some extent, 


Table 5.2: Scale Scoiesand Factor Loadings for Organizational Outcomes Scales 


Organizational Outcomes - Scales a 

Factor Loading 

Cronbach 

a 

M (SD) b 

Departmental Change 


0.92 

2.39 (1.09) 

-Led to changes in teaching practices in my department 

0.90 



-Led to curricular changes in my department 

0.93 



-Led to changes in educational values in my department 

0.90 



-Informed departmental strategic planning 

0.80 



Institutional Change 

0.77 

0.95 

2.05 (0.97) 

-Led to developing communities of practice at institution 

0.96 



-Led to developing campus network for STEM reform 

0.95 



-Led to curricular changes in other departments 

0.91 



-Led to emergence of new campus leaders for change 

0.82 



-Led to changes in teaching in other departments 

0.80 



-Led to changes in educational values in other departments 

0.74 



-Informed campus strategic planning 

0.68 



-Led to campus workshops and professional development 

0.66 



NOTE: a Factor loadings and Cronbach a's calculated for total sample only, b Five-point scale with 1 = 
and 5=To a great extent. 

Not at all, 3=To some extent, 


Outcomes and Benefits from Participation | 23 










































Table 5.3: Sc ores for Individual Outcomes 


Individual Outcomes - Individual Items 

M (SD) a 

Recharged and/or energized in work 

3.27 (1.25) 

Allowed to have fun in professional environment 

3.10 (1.28) 

Lent credibility for approach to teaching 

3.01 (1.33) 

Lent credibility for approach to professional work 

2.84(1.34) 

Afforded opportunities to collaborate on projects 

2.58 (1.35) 

Assisted in career advancement 

2.30 (1.30) 

Gave opportunity to pursue new grants or major projects 

2.28 (1.33) 

Led to publications 

1.72 (1.15) 

Gave skills to make transition from faculty to administration 

1.88 (1.26) 

NOTE: a Five-point scale with 1 =Not at all, 3=To some extent, and 5=To a great extent. 


Individual Benefits 


Figure 5.1 shows the frequency of benefits through community involvement cited by the participants. In general, our 
participants reported the greatest benefits of involvement in these communities coming in the form of learning and 
improving their practice, reenergizing them in their satisfaction and fulfillment in their work, and gaining credibility 
for their work related to STEM reform. These individual benefits are important for different reasons among faculty 
engaging in STEM reform, and we delve deeper into these outcomes below. 

LEARNING AND IMPROVING PRACUCE 

Of the three broad outcomes we mentioned above, learning and improving practice was the most frequently cited 
benefit of community involvement (67.3%). This is encouraging for the work of STEM reformers, as more than two 
in three community members indicated that they changed their teaching and improved in practice as a result of their 
involvement in these communities. Much of the work of these communities is focuses on pedagogical reform, which 
comes in the form of applying specific methods in the classroom (such as process-oriented, guided-inquiry learning) 
or adopting a new perspective or approach to science education (such as studying science through complex issues 
like HIV and climate change). When we asked participants in our interviews to identify the most important aspect 
of their community involvement, faculty would often cite the pedagogical approach to science education in their 
community. As one of the primary foci of STEM reform remains getting faculty to adopt evidence-based teaching 
practices (Dancy & Henderson, 2008), this finding points to the value of these communities in these efforts. How¬ 
ever, effective STEM reform requires more than simply engaging faculty in improving teaching; the other frequently 
cited benefits in this study highlight other ways in which these communities contribute to STEM reform. 

REENERGIZING FAC ULTY IN THEIR WORK 

Feeling reenergized and having fun in a professional environment are the two most frequently cited benefits of com¬ 
munity involvement (73.7% and 69.2%, respectively). STEM reform can be difficult work, especially for faculty seek¬ 
ing to change their teaching practice. They face many barriers to making meaningful changes, including institutional 
reward structures that undervalue innovation in teaching, disciplinary cultures that place more emphasis on research, 
and a lack of institutional leadership that fosters a culture that values teaching (Austin, 2011; Henderson, Beach, & 
Finkelstein, 2011). Faculty who value teaching and are interested in student learning may reach a point where they 
lack the energy or motivation to keep pursuing this work in the face of these barriers. Our interviews and observa¬ 
tions revealed that faculty felt renewed in their efforts due to the supportive and innovative spaces of these communi- 
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NOTE: Benefits of involvement were measured on a 5-point scale, with 1 = Not at all; 3 =To some extent; 5 =To a great extent. Values in 
chart indicate the frequency of participants who indicated at least a score of 3 for these benefits. 



ties. Many spoke of the difficulties of engaging in reform in institutions that were less supportive, and they were able 
not only to develop or regain a passion for this work, but they also looked forward to the chance to let loose and have 
fun. The role these communities play in helping faculty tap into this passion and sense of fun is vital to their success. 

LENDING CREDIBILITY TO FACULTY AND THEIR REFORM WORK 

Faculty participants also indicated that they gained credibility for their approaches to teaching (64.7%) and profes¬ 
sional work (59.5%) due to their community involvement. As we mentioned above, institutional and departmental 
structures in which STEM faculty often find themselves can tend to focus on research over teaching and student 
outcomes. Faculty who are interested in such educational aspects of their work may feel that they are seen as less 
credible due to these interests. The communities in our study work hard to engage faculty members in this meaning¬ 
ful work, and they also are attuned to the structures that can inhibit faculty in these pursuits. Leaders of these com¬ 
munities play key roles in supporting faculty members who are actively involved, even going so far as writing letters 
of support for tenure and promotions and communicating with community members’ institutions about their accom¬ 
plishments and how their work contributes to broader progress in STEM reform. These communities also serve as 
the primary home for professional development for many faculty in our study, and the nature of the work and value 
communicated by the leadership of these communities go a long way in supporting faculty in pursing these efforts. 


Organizational Outcomes 


While less prevalent, we found that participants also indicated some organizational benefits from their involvement 
in these communities. Nearly 35% of participants indicated that engagement in these communities contributed to 
some extent to changes related to STEM reform in their departments, while more than one in five of the participants 
indicated that some sort of institutional change had come about as a result of their involvement in these communi¬ 
ties. We interviewed several individuals from a variety of institutions who were able to infuse the practices from one 
of the communities into their departments on campus. This often occurred when multiple members from an institu¬ 
tion were engaged in the community, especially when this included administrative leadership (e.g., department chairs, 
deans). We found that departments and even small institutions or schools of science utilized the principles of the 
communities to guide curricular and strategic planning for their science education efforts. The United States Military 
Academy at West Point and Brigham Young University Idaho are two good examples of institutions that utilized the 
SENCER Ideals (reproduced in Appendix 4A) as a guide for science courses throughout their curricula. 

We have seen how the broad aggregate results of our research point to the benefits that communities of transfor¬ 
mation can provide to STEM reform. We will now highlight several findings from our multivariate analyses that 
suggest particular reasons why administrators and faculty might gain from engagement with communities of trans¬ 
formation such as the ones in our study. 


Multiple Benefits of Ongoing Involvement 


For faculty members who pursue ongoing support for teaching and professional development, these communities of 
transformation offer a highly meaningful experience. Our analyses showed that faculty who were more continuously 
involved in these communities (i.e., attended more events and engaged more frequently with the community) exhib¬ 
ited greater benefits related to learning to improve practice, leadership skills, and networking (See Table 6.3). Our 
interviews and observations revealed that faculty who continuously engaged with these communities enjoyed ongo¬ 
ing support from the leadership of the CoTs and often reconnected with their own passion for teaching and for their 
disciplinary work. 

This dovetails with the fact that feeling reenergized in one’s work was the most prevalent benefit cited by faculty in 
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our surveys. 


The benefits from community involvement we examined—leadership skills, improving practice, networking, grant 
opportunities, opportunities for career advancement, and others—are likely attractive to many faculty members. 
When we examined motivations for faculty to become involved in these communities, we identified some important 
patterns. Faculty in our study cited a host of reasons that motivated them to join these communities. While some of 
these faculty were oriented toward a specific kind of benefit, such as the desire to improve teaching, we found that 
participants benefited in several outcomes even when they were not seeking those particular benefits. For example, 
faculty seeking to improve their teaching practices (the most oft-cited motivation for involvement) not only reported 
greater benefits in learning and improving practice, but also reported greater benefits relating to leadership develop¬ 
ment and networking. In fact, higher scores for all five of our broad benefit constructs resulted among participants 
who identified any of the following motivations for involvement: support for change on campus, strategies to involve 
peers in change, opportunities for grant funding, and connecting to like-minded colleagues. This suggests that par¬ 
ticipation can lead to multiple benefits beyond one’s initial reasons for engaging in these communities. 

Our qualitative interviews provided a good example of these expanding benefits of involvement. One faculty member 
described entering a community initially because she was interested in improving an introductory course, but through 
her involvement she began to engage colleagues in rethinking the departmental curriculum. As a result of her passion 
for improving teaching and her knowledge gained through involvement in the community, she moved into a role in 
the Center for Teaching and Learning. The initial benefits pushed her into new and more effective areas over time. 
This story is an example of a common narrative arc within our interviews, which was mirrored broadly in our survey 
results. 


Who Benefits? GainsforWomen and Faculty of Color 


By comparing reported benefits according to two characteristics particularly important to STEM reform—gender 
and race/ethnicity—we uncovered some significant trends (see Table 5.4). For nearly all of the individual benefits we 
studied (except for involvement leading to publications), female faculty members reported statistically significantly 
greater benefits resulting from their participation in these communities as compared to their male counterparts. 
While all such disproportionate benefits are notable, we are especially drawn to the difference between men and 
women reporting improvement in skills for leadership and change, gaining skills to transition from faculty to admin¬ 
istration, and contributing to career advancement. In a similar fashion, faculty of color indicated greater benefits than 
White faculty members in several key benefits—networking, being afforded the opportunity to pursue new grants 
or projects, gaining the opportunity to collaborate with others on projects, gaining credibility for their approach to 
professional work and teaching, and gaining skills to make the transition from faculty work to administration—as a 
result of their involvement with these communities. 

These pronounced benefits for women and faculty of color suggest the importance of communities of transformation 
for these traditionally marginalized populations in STEM fields. With increasing efforts to diversify STEM fields 
and disciplines, these communities seem to provide increased support for these groups of STEM faculty than may 
exist on their home campuses. This is certainly reinforced by our interviews and participant observations throughout 
the project. We observed and were told repeatedly of faculty members feeling supported by both leaders and fellow 
participants in the communities, which frequently served as their main professional communities outside of their 
institutions. Often faculty members expressed much more colleagueship with people in the community than at their 
own institutions, which helped them remain in faculty positions despite the fact that they may have otherwise left 
academe without this support. For faculty of color in particular, the key benefits mentioned above point not only to 
the connections that can be made but also to the benefits that can accrue from these connections through involve¬ 
ment in these communities. Faculty of color gain significant social capital that provides access to grant and research 
opportunities, presentations that provide visibility, and networking/personal connections for letters and promotion. 
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Table 5.4: Outcome Variable Mean Comparisons by Genderand Race/Ethnicity 


Individual Outcomes - Individual Items 

Female 

M (SD) a 

Male 

M (SD) 

t Statistic 

Effect Size 
Cohen’s d 

Learning and Improving Practice Scale 

3.47(1.00) 

3.18(1.08) 

6.50*** 

0.28 

Leadership/Change Scale 

2.72 (1.14) 

2.53 (1.11) 

3.61*** 

0.16 

Networking 

2.64(1.09 

2.43 (1.12) 

4.35*** 

0.19 

Departmental Change 

2.44(1.09) 

2.34(1.10) 

2.07* 

0.09 

Institutional Change 

2.08 (1.09) 

2.02 (0.96) 

1.16 


Recharged and/or energized in work 

3.45 (1.20) 

3.05 (1.27) 

7.78*** 

0.33 

Allowed to have fun in professional environment 

3.26 (1.24) 

2.90 (1.30) 

6.61*** 

0.28 

Lent credibility for approach to teaching 

3.16(1.32) 

2.83 (1.32) 

5.81*** 

0.25 

Lent credibility for approach to professional work 

3.00 (1.32) 

2.67(1.33) 

5.74*** 

0.25 

Afforded opportunities to collaborate on projects 

2.68 (1.36) 

2.48 (1.32) 

3.52*** 

0.15 

Assisted in career advancement 

2.39 (1.34) 

2.20 (1.25) 

3.46** 

0.15 

Gave opportunity to pursue new grants or major projects 

2.38 (1.35) 

2.17(1.28) 

3.75*** 

0.15 

Led to publications 

1.74(1.18) 

1.69(1.12) 

0.97 


Gave skills to make transition from faculty to administra¬ 
tion 

1.95 (1.30) 

1.81 (1.22) 

2.26* 

0.11 


White 

M (SD) 

FOC b 

M (SD) 

t Statistic 

Effect Size 
Cohen’s d 

Learning and Improving Practice Scale 

3.32 (1.05) 

3.39 (1.04) 

1.11 


Leadership/Change Scale 

2.62(1.13) 

2.72(1.17) 

1.31 


Networking 

2.51 (1.09) 

2.73 (1.15) 

3.19** 

0.14 

Departmental Change 

2.38 (1.08) 

2.44(1.14) 

0.90 


Institutional Change 

2.04 (0.95) 

2.13 (1.05) 

1.32 


Recharged and/or energized in work 

3.26(1.25) 

3.29 (1.24) 

0.31 


Allowed to have fun in professional environment 

3.11 (1.28) 

3.07(1.30) 

0.54 


Lent credibility for approach to teaching 

3.00 (1.33) 

3.13 (1.33) 

1.68+ 

0.07 

Lent credibility for approach to professional work 

2.82(1.34) 

3.00 (1.33) 

2.32* 

0.10 

Afforded opportunities to collaborate on projects 

2.57 (1.34) 

2.71 (1.36) 

1.78+ 

0.08 

Assisted in career advancement 

2.29 (1.29) 

2.40 (1.35) 

1.48 


Gave opportunity to pursue new grants or major projects 

2.24(1.32) 

2.51 (1.36) 

3.36** 

0.15 

Led to publications 

1.71 (1.15) 

1.80(1.18) 

1.24 


Gave skills to make transition from faculty to administra¬ 
tion 

1.85 (1.25) 

2.07(1.29) 

2.48* 

0.12 


NOTE:+p < .10; *p < .05; **p < .01; ***p < .001; a Five-point scale with 1-Not at all, 3-To some extent, and 5-To a 
great extent; b FOC-Faculty of Color 
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Team Involvement for Organizational Change and Leadership 
Development 


We also identified how the involvement of several individuals from a single institution increased departmental and 
institutional benefits, as well as boosting reported individual benefits for developing skills for leadership and change 
(See Table 6.3). In fact, one of the largest effect sizes we observed in our analyses in predicting departmental and 
institutional benefits came from having more peers from a single institution also involved in the community. Some 
of the communities of transformation in our study intentionally harness this effect through a campus team structure, 
in which campuses are encouraged to send teams of faculty and administrators to events to engage in group learn¬ 
ing and reform. Yet, even in the communities that are not based on this team structure, we still observed benefits of 
involvement of multiple faculty members from the same institution. 

By encouraging multiple faculty members from the same institution to pursue engagement in the same community 
of transformation, administrators can foster a common language and strategy of reform among their faculty mem¬ 
bers, which our findings suggest contribute to greater departmental and institutional change. These team members 
can learn together from other members of the community and gain ideas for strategies to apply to their home cam¬ 
puses. Additionally, faculty reported greater individual benefits in leadership and change skills when more of their 
peers were involved in the community, suggesting that being able to engage with one’s colleagues in these reform 
communities increases the ability to lead on their home campuses. 


Additional Strategies for Broader Impact/Outcomes 


While our study focused on the effects on individuals and how individuals influenced their departments and institu¬ 
tions, our qualitative data demonstrated further impact that we were not able to capture on the survey. In this section, 
we describe some of these important strategies/impacts that can result from supporting large scale CoPs. Section 
9, in which we elaborate on the maturing phase of the CoP lifecycle and the focus on community expansion, also 
demonstrates some of the broader impacts that can be achieved by reshaping disciplinary groups, influencing larger 
institutional environments and whole sectors, targeting regional areas through regional networks, and even increas¬ 
ing international efforts through work and partnerships abroad. In this section we continue to describe the significant 
impacts of these CoPs that are often overlooked. 

CREAUNG OTHER NETWORKS OR COMMUNITIES OF PRACTICE 

Some of the communities in this study have fostered the development or creation of other networks or CoPs for 
improving STEM education. PKAL in particular was responsible for establishing many other networks and com¬ 
munities of practice such as the Faculty for Undergraduate Neuroscience (FUN), the National Numeracy Network 
(NNN), and Quantitative Inquiry, Reasoning, and Knowledge (QuIRK). SENCER similarly works to create a net¬ 
work of informal educators focused on making science education more relevant. 

WORKING AC ROSS AND BETWEEN COMMUNITIES OF TRANSFORMATION 

Over time, the four communities of transformation have each co-hosted events with other STEM reform groups 
in an effort to broaden their impact by having more individuals attend. PKAL and BioQUEST have hosted several 
events together, including the BioQUEST summer workshop that we attended, which was hosted in conjunction 
with a PKAL regional meeting. 

NATIONAL REPORTS AND ACUVmES 

These CoTs were also very involved with the creation of national reports and activities aimed at STEM reform. 
BioQUEST’s involvement with BIO 2010 (National Research Council, 2003) is an example of working with national 
organizations on a report aimed at creating large-scale pedagogical and curricular changes. Leaders in BioQUEST 
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served on the planning groups, and they were instrumental in providing ideas stemming from their own philosophy 
and curricular materials. BioQUEST was also instrumental in revising the MCAT. 

A CADRE OF LEADERS 

PKAL’s Faculty for the 21st Century (F21) program was directly aimed at creating a set of leaders among STEM 
faculty that would advance into department chair, dean, and other leadership roles on campuses that would help 
to propel needed reforms over time. Recognizing that many STEM faculty neither aspire nor are prepared to play 
leadership roles, the program helped inspire and train these individuals. They were able to explore their passions in 
order to identify areas in which they might want to take leadership. These faculty were also provided training on how 
to manage departmental politics, how to create a shared vision, how to overcome challenges to creating change, and 
other skills important to implementing these important STEM reforms. 

SERVICE AS CONVENERS AC ROSS GROUPS 

Particularly in the early years, there was little communication among different STEM reform efforts; the CoTs in our 
study played a pivotal role in convening related efforts and bringing together intermediary groups that could syn- 
ergize their shared work. On numerous occasions, PKAL served as a convener by hosting science education groups 
across disciplines and associations. 

CONSULTANCIES 

Each of these CoTs offered different types of consultancies over the course of their histories to support other insti¬ 
tutions or groups in STEM reform efforts. PKAL offered consultancies, supported by the Keck Foundation, that 
helped institutions pursue particular STEM reform projects, ranging from facilities reform to faculty and leadership 
development. SENCER offered “house calls”where SENCER leaders went out to campuses to help with curricular 
reform efforts and to coalesce faculty to work across different departments in support of new curricula. The POGIL 
Project provided consultancies for campuses that were interested in integrating the POGIL Project activities across a 
department or a set of departments. 

Our study did not examine the relationship between certain design features and these broader educational impacts, 
but these CoTs seem to offer compelling examples for motivating this kind of change. PKAL, for example, appeared 
to be involved in activities that related to some of these broader impacts to their work on leadership development, 
to the creation of networks, and to the importance of convening. These areas all suggest directions that other future 
CoTs might want to explore if they seek to have these broader, enterprise impacts. 


Summary 


This section describes the outcomes and benefits of participation in STEM reform CoTs, while also highlighting 
additional strategies for achieving broader outcomes for STEM reform. Our findings reveal that these communities 
contribute most frequently to individual outcomes related to feeling reenergized in one’s work, learning and improv¬ 
ing practice, and gaining credibility for one’s work, yet they also contribute to broader outcomes for faculty members’ 
departments and institutions in terms of broader curricular and pedagogical uptake. These communities have also 
contributed to broader change through other strategies we highlight, although it is hard to capture these impacts em¬ 
pirically. We turn in section 6 to the ways in which these communities are designed to contribute to these outcomes 
and benefits. 
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VI. Design for Engagement & Outcomes 


O ne of our central research questions was: How do members and leaders of communities of practice 
(CoPs) perceive CoP design (membership, structure, communication, activities, and organization to 
support new knowledge development and action) shapes the ability to achieve goals (around under¬ 
graduate STEM pedagogical change and diffusion)? 

We utilized both qualitative and quantitative data to answer this question as we examined how these communities 
are designed to facilitate engagement and contribute to outcomes. Our findings related to the ways that communities 
of transformation (CoTs) are designed to engage faculty and accomplish their outcomes, and we found this to be dis¬ 
tinctive from the literature on communities of practice and social networks. Therefore, the findings below represent 
distinct and new ways to think about designing STEM reform communities to be successful. These findings are also 
supplemented by section 7, on the formation and lifecycle of CoTs, which reviews how these communities developed 
the engaging philosophies and communities described here. These two sections together are meant to help readers 
understand how CoTs can be structure to best meet their goals. 


Communities of Transformation as Unique and Important Professional 
Development Opportunities 


Before describing how these communities are designed for engagement and the achievement of outcomes, it is useful 
to compare the opportunities presented by CoTs to other professional development opportunities available to faculty, 
such as conferences or workshops organized by disciplinary societies or on-campus professional development re¬ 
sources. This comparison can show some of the potential advantages of CoTs. 

In disciplinary societies, research is often the main focus of professional development. Typically, these societies bring 
together large numbers of faculty and have few venues for more intimate interactions that lead to deep learning. In 
contrast, the reform communities of transformation in this study focus on educational outcomes, particularly as they 
arise from developing faculty as teachers and contributing to individuals’ efficacy for leadership and change. These 
communities engage in faculty development more intentionally, drawing on research on teaching and learning that 
is ignored by many traditional forms of professional development. Also, as a result of the interpersonal component 
of these CoTs, these communities provide ongoing support for faculty members over the course of their careers—we 
noted that many of the communities we studied would provide letters of support for faculty in their tenure and pro¬ 
motion processes. 

Among on-campus professional development opportunities, Centers for Teaching and Learning often offer train¬ 
ing around the latest development or issue in pedagogy (e.g., problem-based learning), but this training may not be 
aligned with faculty members’interests. Most such teaching development is practical in orientation. In contrast, the 
CoTs in this study provided teaching-oriented professional development that was decidedly philosophical in orienta¬ 
tion; participants gained a deep connection to this new approach to teaching, something impossible through a simple 
skill-development session. Because the CoTs were founded on a specific philosophy toward STEM reform and 
teaching, they provide faculty with sustained, focused development opportunities, rather than ones where the focus 
constantly changes. 

These communities also differ in the level of engagement and trust developed through involvement in them. As we 
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mentioned earlier, the benefits cited in conjunction with our interview and observation data suggest that these com¬ 
munities are incredibly fulfilling, both personally and professionally Individuals indicate this sense of fulfillment in 
their descriptions of the trusting relationships that are formed through their involvement. A defining feature of these 
communities is the ongoing opportunity to develop relationships in intimate settings, through intentional engage¬ 
ment with both leadership and community members. This shines through in the trust that participants describe as 
operative within these communities. In CoTs, engagement is modulated across the various sizes of gatherings, offer¬ 
ing faculty opportunities to participate in both large group and more intimate settings. 

In Table 6.1, we outline these key differences we identified among professional development opportunities through 
campus-based efforts, disciplinary/professional societies, and the communities of transformation we studied. These 
differences, which were not only communicated to us by community members but also observed by us through 
participant observations at community meetings and events, suggest some of the ways in which these communities 
benefit individual faculty participants and their home departments and institutions. 


Designing for Engagement 


These four CoTs offered a wide variety of ways to engage individuals, including workshops, newsletters, social media, 
presentations at disciplinary conferences, websites, resources, and publications. Two design facets were most influ¬ 
ential in contributing to participants’sense of engagement with the community: 1) the philosophy, which was most 
often epitomized through a signature event; and 2) relationships, which were formed through peer-to-peer learning, 
brainstorming with others engaged in similar STEM reform efforts, and opportunities for mentoring. 

We captured our findings about how to design for engagement and outcomes in the model in Figure 6.1. The central 
and most important aspect for these communities was the philosophy, with all the other elements emerging out of 
this core. Resources, key events, and others are all important as they embody this philosophy. Thus, CoTs for STEM 
reform embody their philosophy of pedagogy in their activities, resources, communications, and all other activities. 
The next circle is personal interactions that progress from peer-to peer learning to more focused brainstorming to 
formal mentoring. We begin by describing the themes that emerged from the first phase of qualitative data collec¬ 
tion, followed by general findings from the survey portion of the study. 


Figure 6.1 

Model of Core Design Characteristics for 
Engagement in STEM Communities of 
Practice 
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Table 6.1 

Comparison oflypical Professional Development through Campus-Based Opportunities, 
Disc iplina ry/ Professiona I Societies, and Communities of Transformation 



Campus-Based 
Opportunity 

Disc iplina ry/ Professiona 1 
Society 

Community of 
Transformation 

Nature of 
Involvement 

Topic of focus tends 
to rotate (although 
some campuses have 
professional learning 
communities on a topic); 
involvement is occasional 
and not on-going; weak 
relationships formed 

Annual involvement 
through meetings; typically 
weak relationships formed 
through participation 

Involvement through a mix 
of annual events that bring 
people together for an ex¬ 
tended period of time, with 
combination of large and 
intimate spaces within those 
meetings and a mix of ongo¬ 
ing communications and 
involvement throughout the 
year; potential for building 
stronger relationships 

Focus 

Focused on teaching 
more generally 

Focused almost exclusively 
on research within a 
discipline 

Focused on teaching and 
skill-building, utilizing 
a cohesive philosophical 
approach across experiences; 
community forms around 
this philosophy 

Size 

Typically an intimate 
size, which can be 
intimidating to people 
who want an introduction 
or are not sure about the 
involvement 

Overall meetings typically 
large and impersonal; those 
focused on teaching have 
few places to gather 

Mix of large venues 
that allow networking 
with national leaders, as 
well as small spaces to 
interact; allows a space for 
newcomers as well as more 
advanced participants to 
engage 

Engagement 

Dyadic; sometimes a 
more passive learning 
style 

Typically a passive approach 
to discussing teaching, such 
as a workshop or p re- 
conference event 

Highly interactive; 
collaborative and active 
learning approaches 

Trust among 
Participants 

Trust is difficult to de¬ 
velop through one-off 
workshops and trainings 

Trust is typically lacking, 
with few opportunities to 
development meaningful 
connections 

Trust frequently developed 
through intimate settings, 
targeted interactions, and 
connections that are able to 
grow over time 
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Philosophy as Design 


When we asked faculty what they thought was the best or the most meaningful aspect of the CoT in which they 
participated, they described the philosophy of the community, which they thought was best symbolized or actualized 
through the signature events of each community. However, they also noted that philosophy was embedded in most 
of the community activities and communication. Our data about the formation of these communities demonstrate 
how much leaders invested in carefully constructing a compelling philosophy, typically articulated through a seminal 
document, posted on their websites, and reiterated throughout their materials. Each community then spent signifi¬ 
cant time embedding the philosophy in the activities and work of the CoT. This speaks to the intentionality invested 
by leaders within these CoTs to craft a philosophy that would be pervasive to participant experiences. The leaders of 
these CoTs also talked about ways they personally believed in and worked to spread the philosophy. We present the 
ways in which each community enacts its philosophy below as the best evidence of this theme. 

THE POGIL PROJECT 

For the POGIL Project, as participants described workshops and the annual national meeting, they noted how the 
meetings utilize the principles of the specific the POGIL Project pedagogy (active learning through guided inquiry 
and process features, like teamwork), which means that meetings include very active sessions, are learner centered, 
and use process principles, such as assessment and group work. In addition to using the specific group-work process, 
each session at the POGIL Project’s meetings ends with the specific the POGIL Project approach to assessment, 
dubbed SII (Strengths, Improvement, Insights). As a result, the philosophy that undergirds the STEM reform pro¬ 
moted by the POGIL Project also is reflected in the way that members of the community interact with one another 
at events and through communications. Additionally, the physical spaces in which all of the meetings and events take 
place also reflect this philosophy. The POGIL Project organizes its events so that sessions are always in rooms with 
movable tables and chairs, boards and other working spaces, and technologies that enable more active learning. 

SENCER 

SENCER’s philosophy, documented in the SENCER Ideals (see Appendix 4A), focuses on capacious questions, 
context over content, interdisciplinarity, and the connection of science to civic issues and relevant problems. Making 
meaning, pursuing intellectual curiosity, and fostering a democratic sense of openness are additional strong values 
held as a part of the philosophy of SENCER. We witnessed each of these characteristics active in the SENCER 
annual conference. Each of the speakers alluded to questions of meaning and important problems that need to be 
resolved through science, and leaders implored people often to reflect about the higher purposes of education. One 
participant described how the philosophy resonated and permeated activities, events, and communication: “They 
bring a fundamental epistemology that’s very different than a lot of groups of people that I know to all their work. 
They really have this much more phenomenological approach to teaching, which is kind of an awareness of the meth¬ 
ods they’re using in teaching and just a willingness to think about values, and mission, and care, and love, and the 
things that actually make us more and more human. They can talk about things like truth, beauty, and goodness with¬ 
out it being seen as something polar to the very materialistic approaches of Western science.” Reflection on meaning 
and questioning was built into each of the sessions. The annual event took place at a picturesque university on the 
west coast, where there are a variety of gardens, atria, and benches—all areas for reflection. The events reflected the 
interdisciplinary philosophy by not only inviting faculty across disciplines, but also by using sessions consistently to 
discuss the need to work across disciplinary boundaries. Also, educators from non-traditional groups often considered 
outsiders in academe, whether informal educators (e.g., museum staff) or part-time instructors, are actively included 
in SENCER discussions, and participants from these groups noted “feeling empowered.” 

PRQJ EC T KALEIDOSCOPE 

For PKAL, the What Works documents capture their philosophy of active learning, experiential learning, and ac¬ 
knowledgment of diverse learners’ needs. The Summer Leadership Institute (SLI) epitomizes the focus of their 
philosophy on the importance of developing leaders to enhance STEM reform. Leadership using these principles is 
the mantra of PKAL. The principles related to What Works are seamlessly connected to the activities of the SLI. Over 
the years PKAL has conducted a series of signature events, assemblies, and workshops for their various sub-groups, 
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with each of these events intentionally encapsulating the key principles of “what works” in STEM reform. Further, 
participants spoke about how the underlying philosophy of PKAL includes value of mentoring, relationships, and 
developing people and their careers. As a result, events always include groups of mentors to provide feedback, time to 
build relationships, directed time to meet with mentors, and a focused discussion of careers and ways to develop indi¬ 
viduals. While most of the CoTs have some elements of relationship building and mentoring that we will describe in 
the next section, this was a particularly significant aspect of the philosophy and practices of PKAL. 

BIOQUEST 

BioQUEST’s signature summer workshop embodies its philosophy related to teaching as the process of problem- 
posing, problem-solving, and peer persuasion, known as the Three P’s. The workshop is an opportunity for faculty to 
be problem-solvers, creating new materials to help teach biology. Participants also engage in the role of problem-pos¬ 
ing to understand the types of issues addressed through their problem-solving activities. In this setting, they engage 
in groups to do their work, which involves peer persuasion. Thus, the underlying Three P’s are utilized to shape the 
workshop structure and process. Participants described this approach as being on the cutting edge, future-oriented 
(i.e., oriented toward what will be needed in the future training of scientists), challenging, and creative. BioQUEST 
does not develop prepackaged materials for this work, but expects participants themselves to create new teaching 
materials. The persuasion aspect of this philosophy can also be seen in the almost conflictual dialogue that takes place 
as people debate ideas to move towards more solid pedagogical materials. For example, workshop participants were 
asked to present their emerging ideas to other faculty attending a co-located conference occurring at the same time. 
This forced them to persuade and respond to criticism from relative outsiders who were not as aware of the peda¬ 
gogical approaches or philosophies. Like the POGIL Project, the work spaces for BioQUEST meetings are set up 
to be highly interactive, with a computer for each individual, movable tables and chairs, and small rooms for break¬ 
out sessions or group work. BioQUEST participants often leave the workshop without a finished product, but they 
have learned a process that will involve future problem-posing, problem-solving, and persuasion, which fits into this 
philosophy of STEM reform. 


Personal Interactions: Peer-to-Peer Learning, Brainstorming 
with Others, and Mentoring 


In addition to the pervasive philosophy itself, members mentioned interactions with others in the community as the 
most engaging component of participation in these CoTs. Personal connections/interactions were reported in the 
data over 200 times and were by far the longest report from our hyper-research codes. The main categories of interac¬ 
tion that were mentioned in interviews or observed were: 1. peer-to-peer learning; 2. opportunities to follow up and 
brainstorm about practice with peers and collaborators; and, 3. mentoring. All of these interactions progress toward 
more directed or involved interactions and provide energy for faculty to continue to reform in the face of depart¬ 
ments and institutions that may not be supportive. One participant summed up the sentiment expressed by many as 
follows: “One of the best aspects was simply meeting people that were really impressive people that had lots to share. 
The sort of willingness to share. I would say my personal relationships or personal experiences with individuals in the 
network has been the best thing for me.” 

PEER-TO-PEER MODEL 

Participants noted their own engagement with the overall model of learning from peers, and they highlighted 
the importance of this model for these communities. The peer-to-peer model was evidenced within all four CoTs 
through all of the events we attended and observed. Many people shared that the other opportunities for professional 
development, available through centers for teaching and learning or attendance at disciplinary conferences, often 
lack connection with peers teaching similar courses within similar disciplines. They felt that the information that 
they garnered through the CoT was more directly applicable because it came from peers attempting to do the exact 
same work in similar contexts. The CoTs also emphasize the importance of peer-to-peer learning in person and in 
relatively small settings, while large disciplinary conferences and workshops were mentioned as ineffective settings for 
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developing real relationships. The processes of developing trust and of participants getting to know one another are 
critical if advocacy and mentorship are to play a role in STEM reform. Thus, people emphasized the importance of 
in-person opportunities to build relationships that eventually turned into more detailed brainstorming and mentor¬ 
ing experiences (described below). 

The signature event of each CoT provides ample opportunity for meeting people. The settings for these events are 
often isolated, helping to ensure that participants spend time together. Lengthy introductions are included at the 
beginning of these events, and faculty typically work in pairs and teams to maximize interaction. There are typically 
planned social hours, meals, and field trips to ensure time to bond. 8 Most events include late evening social time 
with wine, games, or both. Many also institute specific programs or policies to ensure that people develop relation¬ 
ships, such as a buddy system between a newcomer and an alumnus/a. At the end of each signature event, people are 
invited to continue contact, and they are told about other opportunities, such as future events, authoring resources, 
joining the listserv, and newsletters. The notion of using the community as a resource is activated in the signature 
events of the CoTs. 

BRAINSTORMING WITH OTHERS 

Faculty also spoke about connecting with people to brainstorm and obtain advice. One participant described leaving 
the first event attended through a CoT and understanding this important ongoing and developing relationship: “You 
were engaging with someone and making a connection [at the event], and chances are, if you actually did go back 
and implement that plan, you could contact those people for help.” Another participant described the role of com¬ 
munities over time: “A sounding board, advisors, mentors. Always a place to go with questions. Okay, it was a safe 
place to go when sometimes it wasn’t safe to be in my department with certain opinions.” Many participants came to 
their first event thinking this would be a one-time interaction, but left with a strong notion that they had a group of 
people that they could now contact for support in their teaching. These more informal connections often developed 
into more formal mentoring relationships over time. 

MEmORING 

Over time, individual phone calls and e-mails among participants turned into more formal mentoring relationships. 
This formal mentoring was integrated into various practices: for PKAL, into its leadership institute and regional 
networks, and for SENCER, into its fellows program and annual dinner. This opportunity for more formal mentor¬ 
ing relationships was mentioned as one of the most valuable aspects intentionally fostered through the communities. 
One participant talked about the roles of mentors: “Really having recruited and picked good people as mentors, and 
having them be available as resources or sounding boards was helpful, faculty always mention that. The mentoring 
was vital to my success.” Designing for engagement means attracting the right types of individuals, as well as creating 
a sense of accountability to give back to the community. Some of the communities even name these individuals; for 
example, in PKAL they were called the village elders. These 30-50 mentors understood that they were part of this 
sub community that had a mentoring responsibility, and they actively enacted this role for extended periods of time, 
some up to 30 years. One of the village elders described how gratifying it is to be a mentor: “Em now mentoring 
probably eight or nine young people, and I keep in contact with them when they’ve got things going on, they brain¬ 
storm with me. And I find that reciprocal mentoring as just wonderful.” Later in the conversation she mentioned 
how the community itself helped mentors to see that they can also learn and be enriched through these interactions. 
Communities can help to foster these mentoring relationships in ways that make them mutually rewarding. These 
mentoring relationships turn into very tangible advice about departmental politics, grants, tenure and promotion, 
publications, and the like. Mentors often wrote letters for grant projects and tenure and promotion files. 


8 For example, SENCER facilitated trips to science and art museums, and PKAL offered hiking in mountains, exploring 
the town, or driving to a nearby national park. 
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Trends in Quantitative Analyses Related to Design 


The survey findings reinforce the importance of philosophy and relationships in contributing to participants’ effec¬ 
tiveness in STEM reform work (See Table 6.2). These results are especially telling considering that the survey was 
administered to a much more representative sample of community members than were represented in the leader¬ 
ship interviews, suggesting the importance of philosophy and culture in these communities. We anticipated that the 
faculty participating in the survey would be less involved with the CoTs than those we interviewed, and that these 
elements of philosophy and personal interactions may be less important to them. However, when participants were 
asked to rate the importance of various design characteristics in contributing to their work, participants on aver¬ 
age ranked items pertaining to philosophy and relationships as more important than other design principles, such 
as communication vehicles, mechanisms for feedback, sub-groups, grant activities, and events. Innovative and new 
ideas and community philosophy are the first and second most important design characteristics for participants. The 
innovative and new ideas disseminated by the CoTs in this study stem directly out of their philosophies, like guided 
inquiry, active learning, or case-based approaches. Participants indicated that the characteristics were also influential 
in contributing to institutional reform efforts. 

The other top-rated design characteristics by-and-large reference the importance of personal interactions in help¬ 
ing participants’effectiveness in STEM reform. These characteristics include opportunities to connect with other faculty 
and STEM leaders, having a safe space, inclusive practices, and the opportunities to be mentored. It is clear that having the 
opportunity to connect with others in an environment that is safe and inclusive, like the environments we observed at 
the community events, carries through to this representative sample. The consistency across the interviews, observa¬ 
tions, and survey data clearly indicate that philosophy and personal relationships and interactions are central compo¬ 
nents for aspiring communities to attend to when pursuing STEM reform. 

In addition to the these analyses, we also utilized ordinary least squares regression models to examine the extent to 
which design characteristics of these communities, along with different types of engagement in these communities, 
were associated with our five outcome factors (learning & improving practice, skill development for leadership & 
change, networking, departmental change, and institutional change). Tables 6.3 and 6.4 display the coefficients for 
these variables in the models. 9 

LEARNING AND IMPROVING PRACUCE 

We begin first with the model for learning and improving practice. Several engagement variables are significantly, 
positively associated with participants perceiving that they learned and improved practice, after controlling for the 
other variables. These are (in order of effect size): indicating greater continuity of involvement, attending more na¬ 
tional events, indicating a longer tenure of involvement with the community, and authoring materials for the com¬ 
munity. Thus, individuals who have been involved in the communities longer and more continuously can be expected 
to exhibit greater benefits in terms of learning and improving practice, while also attending more events and author¬ 
ing more materials. Participants placing greater importance on several design variables is also positively associated 
with the participants perceiving benefits related to learning and improving practice. Specifically, participants who 
have been exposed to and value community philosophy, community leaders, community culture, innovative and new 
ideas, and community resources report greater benefits related to learning. 


9 The beta coefficients indicate the strength of the expected relationship for each variable to the dependent variables. Each 
coefficient represents the portion of a standard deviation increase in the dependent variable for a standard deviation increase 
in each variable. For example, a participant who scores one standard deviation above the mean for extent of involvement is ex¬ 
pected to score .11 standard deviations higher on the dependent variable “Learning 8c Improving Practice.” These tables show 
only the coefficients pertaining to the design and engagement characteristics. We controlled for a variety of other factors in 
the models, including institutional characteristics (e.g., control, size, Carnegie classification), professional characteristics (e.g., 
position classification, discipline, motivation for involvement), and personal demographics (e.g., gender, race/ethnicity). 
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Table 6.2: Importance of Design ftinciples for Individual and Institutional Effectiveness 
in STEM Education Reform a 



Individual 

Institutional 


Mean (SD) 

Mean (SD) 

Innovative and New Ideas 

2.91 (0.78) 

2.58 (0.86) 

Community Philosophy 

2.87 (0.86) 

2.51 (0.91) 

Opportunity to Connect with Other Faculty 

2.79 (0.89) 

2.43 (0.93) 

Opportunity to Network with STEM Leaders 

2.69 (0.94) 

2.49 (0.95) 

Community Culture 

2.66 (0.89) 

2.36 (0.88) 

Safe Space 

2.64 (0.96) 

2.39 (0.95) 

Inclusive Practices 

2.54 (0.94) 

2.27 (0.93) 

Opportunity to be Mentored 

2.49 (1.02) 

2.31 (0.94) 

Community-Specific Resource 

2.45 (0.94) 

2.21 (0.91) 

Community Leaders 

2.45 (1.00) 

2.21 (0.96) 

Different Opportunities for Involvement 

2.44 (0.92) 

2.25 (0.90) 

Seminal Documents 

2.42 (0.94) 

2.32 (0.93) 

Opportunities for Early-Career 

2.38 (1.05) 

2.40 (0.96) 

Opportunities for Mid-Career 

2.37 (0.97) 

2.32 (0.94) 

Heterogeneity of People Involved 

2.37 (0.97) 

2.26 (0.97) 

Presence at Disciplinary Meetings 

2.27 (0.86) 

2.06 (0.82) 

Opportunities for Late-Career 

2.26 (0.99) 

2.19(0.95) 

Communication Strategies 

2.26 (0.86) 

2.12(0.86) 

Sub-Groups/Grant-Related Initiative 

2.25 (1.02) 

2.30 (0.99) 

Local/regional Events 

2.24 (0.94) 

2.11 (0.89) 

Annual Events 

2.13 (0.95) 

1.98 (0.84) 

Mechanisms for Feedback 

2.06 (0.88) 

1.99 (0.87) 

Community Staff 

2.00 (0.99) 

1.87 (0.92) 

NOTE: a Scale: 4-point scale: 1 = Not at all important; 2 = 
important; 4 = Essential 

Somewhat important; 3 = Very 


SKILLS FOR LEADERSHIP AND C HANG E 

Beginning with engagement variables, we find that attending national events, indicating more continuity of involve¬ 
ment, and having more peers from one’s home institution involved in the community are significantly, positively 
associated with this dependent variable. As for design characteristics, participants noting the importance of commu¬ 
nity leaders and community philosophy result in the two largest positive effects for this perceived benefit, followed by 
valuing innovative and new ideas. 
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NETWORKING 

Again, several engagement variables are significantly associated with networking after controlling for other variables 
in the model. Specifically, having more continuous involvement, attending national and regional/local events, and 
authoring materials for the community are positively associated with networking, while having more years of involve¬ 
ment in the community is negatively associated with networking. 10 Turning to design variables, greater emphasis 
placed on community leaders and community culture is positively associated with networking, as well as valuing 
opportunities to connect both with other faculty and with STEM leaders. 

Thus, the nature of faculty engagement in reform efforts matters, but effectiveness of engagement is also tied to im¬ 
portant design characteristics. Our models suggest that several engagement and design variables are important across 
our outcomes. We measured a variety of possible engagement activities, including attendance at a variety of events, 
engagement in different community activities, membership in different groups within the community, and the nature 
and extent of involvement generally in the community, yet we found that continuous involvement and attendance at 
annual events were the most important engagement variables in our study. In addition to engagement, we found that 
culture and community leaders also play a role in contributing to these outcomes. 


Reinforcing the Importance of Philosophy and Interactions 


It makes sense that annual events, continuous involvement, and community culture, philosophy, and leaders were 
significant in our models, given our understanding of how these communities of transformation strategically try to 
impact STEM reform (Table 6.3). Our interviews with key leaders and our observations of these annual, signature 
events revealed the importance of these events in reinforcing the underlying culture and philosophy of these com¬ 
munities. We saw that community leaders and event organizers act intentionally to infuse the community philosophy 
into the content and organization of sessions, and we observed how the culture of each community is present at these 
events in the ways that members interact with one another and with the key leaders. In other words, community is 
genuinely fostered in these environments, and it is influenced by the philosophy communicated by leaders in both 
event content and organization. We also observed individuals who had attended many events engage in deeper en¬ 
gagement and relationship building at these events, behaviors that can be attributed to their continuous involvement 
and ability to remain connected to the community. 


Deeper Engagement Matters for STEM Reform 


The findings related to continuity of involvement, engaging multiple members from a campus in the community, 
and length of involvement suggest the importance of deeper engagement in contributing to the outcomes in our 
study (See Table 6.3). These variables speak to an active engagement, which signifies faculty engaging more deeply, 
for extended periods of time, and with their peers. These communities can foster such active engagement in order to 
influence faculty members and allow them to see benefits from their involvement in these communities. As we de¬ 
scribed earlier in this section, the ways in which these communities differ from traditional professional development 
opportunities can contribute to this deeper engagement. 


10 This seems at first paradoxical, as one might assume that the longer exposure people have to a community the more 
connections they can make and the greater their networks can grow. We think this finding suggests that the benefit of com¬ 
munity involvement in growing one’s professional or personal network likely comes through in early years of involvement, as 
individuals who are looking for support in STEM reform are likely reaching out to others and growing their support network 
for engaging in such work. Longer involvement is positively associated with other outcomes, suggesting that as individuals are 
involved longer in the community, their goals for support likely shift from needing others to gathering knowledge and skills 
for personal growth and institutional change. 
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Table 6.3 

Bigagementand Design C oeffic ients from OLS Regression Models for Three Individual 
Outcomes Related to STEM Education Reform 



Learning & 
Improving 
Practice 

Skills for 
Leadership & 
Change 

Networking 


P 

SE 

P 

SE 

P 

SE 

Engagement and Design Variables 







Characterized Extent of Involvement 

j ^*** 

.03 

.08** 

.03 

jy*** 

.03 

Attend: National Event 

.09** 

.03 

.08** 

.03 

.08** 

.03 

Design: Community Leaders 

io*** 

.03 


.03 

.15*** 

.03 

Design: Innovative & New Ideas 


.03 

.06* 

.01 

.01 

.03 

Design: Community Philosophy 

.15*** 

.03 

.12** 

.04 

.06 

.03 

Design: Community Culture 

.09** 

.03 

.07 

.04 

.13*** 

.04 

Activity: Author Material 

.05* 

.02 

.02 

.03 

.05* 

.02 

Years Involved with Community 

.08** 

.03 

.04 

.03 

-.09** 

.03 

Design: Community Resources 

.08** 

.02 

.03 

.03 

-.01 

.02 

Number of Peers Involved with Community 

.00 

.02 

.05* 

.02 

.04 

.02 

Attend: Regional/Local Event 

.01 

.02 

.03 

.02 

J J*** 

.02 

Design: Connection with Other Faculty 

.01 

.03 

.00 

.04 

.11** 

.03 

Design: Connection with STEM Leaders 

.04 

.03 

.03 

.03 

.09** 

.03 

Activity: Present at Community Event 

.00 

.03 

.01 

.03 

.03 

.03 

Activity: Present Material at Prof. Meeting 

.03 

.02 

.02 

.03 

.03 

.03 

Activity: Publish about Community Work 

-.02 

.02 

.00 

.02 

.01 

.02 

Group: Leadership/Board Member 

-.07 

.08 

.07 

.09 

.04 

.08 

Group: Project/Grant-Based 

.04 

.05 

.04 

.06 

.07 

.06 

Design: Different Involvement Opportunities 

.04 

.03 

.02 

.03 

.00 

.03 

Design: Safe, Supportive Space 

.04 

.03 

.02 

.03 

.01 

.03 

Design: Inclusive Practices 

.00 

.03 

.02 

.04 

-.01 

.03 

Design: Opportunity for Mentoring 

.04 

.03 

.04 

.03 

.03 

.03 

R 2 

.596 

.528 

.587 


NOTE: *p < .05; ** p < .01; ***p < .001; Variables were standardized prior to entering them in these models, which 
results in the coefficients representing effect sizes. 
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Community Engagement and Design Matters mo re than Institutional, 
Professional, and Personal Characteristics 


While we were able to observe significant effects in their relationships to our outcomes, the same was not true by- 
and-large for the other factors that influence faculty behavior in our models. While we noticed an occasional signifi¬ 
cant coefficient scattered throughout our models, in general a participant’s institutional type, discipline, and rank/ 
appointment status were not significantly associated with the outcomes we measured, when accounting for our other 
engagement, design, and motivation variables. 11 We think this points to the potential for CoTs resembling those in 
our study to contribute to overcoming some of the typical barriers to reform mentioned in the literature, including 
institutional reward structures or policies that value research productivity over improving teaching, disciplinary cul¬ 
tures that place more emphasis on research, and lack of institutional leadership toward a culture that values teaching 
(Austin, 2011; Henderson et al., 2011). While it is true that institutions have complex policies and structures that are 
difficult to change and address—such as those related to tenure, promotion, and rank—and disciplines exert control 
over how faculty may engage in STEM reform, we observed no meaningful differences across these indicators in our 
models. For the most part, regardless of where faculty work, their position, or their discipline, they reported greater 
benefits related to teaching, leadership, and networking through engaging with the important aspects of these com¬ 
munities, and they expressed motivation to attain these benefits. While we can only speculate on the future impact of 
these communities, their efforts could very well contribute to the bottom-up change that scholars in the field identify 
as possible through faculty engagement (e.g., Sunal et al., 2001). We turn now to the organizational outcomes. 


Organizational Outcomes Related to STEM Reform 


After controlling for institutional, professional, and personal characteristics, we identified several participant involve¬ 
ment and CoT design variables that are significantly associated with departmental change (See Table 6.4). Three as¬ 
pects of involvement and activities are positively and significantly associated with participants reporting departmental 
change due to their involvement in the CoTs: length of involvement, peer involvement, and presenting materials re¬ 
lated to community involvement at professional/disciplinary meetings and conferences. In addition to these involve¬ 
ment variables, the salience of three community design variables is also positively and significantly associated with 
departmental changes—community culture, community leaders, and innovative and new ideas of the community. We 
observed that community involvement and design shared many of the same relationships with institutional change as 
it did with departmental change. Length of involvement, greater peer involvement, and presenting materials related 
to community involvement at professional/disciplinary meetings and conferences are all positively associated with 
institutional change, after controlling for institutional and faculty characteristics. However, not all involvement expe¬ 
riences have positive effects; for example, involvement in a project or grant-based group is negatively associated with 
participants reporting institutional change. Of our design variables, participants perceiving community culture and 
community leaders as important for their work is positively associated with institutional change. 


Deeper Engagement Matters for STEM Reform 


Prolonged involvement and presenting about the community to outside audiences were both associated positively 
with our outcomes. These two variables suggest that a deeper engagement in these CoTs can have benefits for 
individuals hoping to influence broader institutional goals for STEM reform, in addition to the individual benefits 
highlighted above (See Table 6.4). In order to present about a community’s pedagogical or other reform strategy 


11 In this report, we only highlight relevant findings related to the communities. For more detailed analyses including insti¬ 
tutional type, discipline, and rank/appointment status, please see Gehrke & Kezar, 2015. 
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Table 6.4 

Bigagementand Design Coefficients from OLS Regression Models for Departmental and Institutional Change 
Related to STEM Reform 


Departmental Change 

Institutional Change 


P 

SE 

P 

SE 

Engagement and Design Variables 


Years Involved with Community 

.15*** 

.04 

.13*** 

.04 

Number of Peers Involved with Community 

.12*** 

.03 


.03 

Activity: Present Material at Prof. Meeting 

.11** 

.03 

.10** 

.03 

Design: Community Leaders 

.09* 

.04 

.12** 

.04 

Design: Community Culture 

.13** 

.05 

.13** 

.05 

Design: Innovative & New Ideas 

.09* 

.04 

.04 

.04 

Group: Project/Grant-Based 

-.07 

.07 

-.17* 

.07 

Characterized Extent of Involvement 

.05 

.03 

-.05 

.04 

Attend: National Event 

.06 

.03 

.02 

.03 

Attend: Regional/Local Event 

.05 

.03 

.06 

.03 

Activity: Present at Community Event 

-.01 

.04 

.07 

.04 

Activity: Author Material 

.04 

.03 

.01 

.03 

Activity: Publish about Community Work 

-.04 

.03 

.01 

.03 

Group: Leadership/Board Member 

-.01 

.11 

-.15 

.10 

Design: Different Involvement Opportunities 

-.01 

.04 

.03 

.04 

Design: Community Resources 

.03 

.03 

.02 

.03 

Design: Safe, Supportive Space 

.03 

.04 

.06 

.04 

Design: Inclusive Practices 

.01 

.04 

.01 

.04 

Design: Connection with Other Faculty 

-.04 

.05 

-.03 

.05 

Design: Opportunity for Mentoring 

-.01 

.04 

.04 

.04 

Design: Connection with STEM Leaders 

.07 

.04 

-.02 

.05 

Design: Community Philosophy 

.04 

.05 

.07 

.05 

TOTAL R 2 

.425 

.488 


NOTE: *p < .05; ** p < .01; *** p < .001; Variables were standardized prior to entering them in these models, which results 
in the coefficients representing effect sizes. 
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to outside audiences, a faculty member must engage long enough and deeply enough to feel comfortable with the 
material and be able to communicate the strategy’s nuances to audiences less familiar with the work. These types of 
presentation also communicate an individual’s expertise, which can translate to increased legitimacy and leadership 
on the individual’s home campus. This in turn can help these individuals to foster more change through their ef¬ 
forts. Additionally, the fact that prolonged involvement is positively associated with the reported institutional change 
outcomes reinforces that STEM reform is complex and takes time (Austin, 2011; Henderson et al., 2011). Designers 
and participants of these and other future CoTs cannot labor under the false impression that brief engagement in 
these communities will lead to larger changes; our findings indicate that those who are involved for longer periods 
of time report greater change. This suggests that such efforts must persist long enough to engage faculty over longer 
periods of time to see these changes take hold. 


Collective Effort for STEM Reform 


Our other key engagement finding points to the idea that change is a collective effort, as faculty who reported having 
more peers involved in the same community with them reported greater change on both the departmental and 
institutional levels (See Table 6.4). The communities in our study can serve as venues for multiple members of an 
institution to engage together in activities related to STEM reform, and the efficacy of such arrangements was 
reinforced by qualitative data. We observed this as a successful strategy in particular when CoTs encouraged mem¬ 
bers to engage in their activities as campus teams. Team members met in the spaces created by the CoTs, and they 
participated in conversations about bringing strategy back to their home campuses and about how they might work 
together when they returned. The qualitative research revealed, however, that such collective action does not only 
occur due to intentional focus by CoTs on gathering together institutional teams. Interviews with active community 
members revealed that there was a positive effect associated with simply having peers at their home institutions who 
had experienced the same workshops or developed through the same community. These commonalities acted to give 
participants a sense of shared language and trust on their campuses, even when they had not participated concur¬ 
rently together with the other individuals. As a result, we conclude that organizational learning within an institution 
is enhanced by having multiple members of that institution participate in the same community. Such contributions to 
institutional reform are often cited as benefits of involvement in communities of practice (Allee, 2000), and the CoT 
model seems to offer this insight into how it can be augmented. 


Community Design for Departmental and Institutional Change 


We also identified several design characteristics of communities of transformation that are positively associated with 
departmental and institutional change (See Table 6.4). Specifically, the presence of key leaders and community cul¬ 
ture are positively related to both institutional and departmental change. Based on our observations and interviews, 
we are not surprised to see the data bear out this association. First, leaders in the community play important roles 
of influence for community members. The four communities in our study were all founded by visionary individuals 
who continuously recruit a cadre of leaders to be active in community operations and generally visible at community 
events. These community leaders exemplify effective leadership styles, which serve as models for community members 
for how to lead change efforts when they return to their campuses. This allows for diffusion of these practices from 
non-organizationally situated communities, such as the CoTs studied, outward to multiple institutions. These lead¬ 
ers also set the tone for community gatherings and events in which faculty learned from one another in peer-to-peer 
settings, fostering cultures of active engagement, trust, and support among community members. 
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Culturesof Personal Support 


These communities also exhibit cultures that value and place personal support at a premium; the presence of such 
cultures are associated with change outcomes on participating faculty members’ campuses. The culture that is fos¬ 
tered in CoTs through community events, resources, and communication materials is a supportive one in which 
faculty are provided valuable feedback and mentorship to help them seek changes on their home campuses. Faculty 
further experience this culture of personal support through the sharing of strategies and best practices, as they learn 
in a peer-to-peer environment (Kezar & Gehrke, 2015a). These findings related to culture should be understood in 
conjunction with the importance of key leaders, as those individuals not only set the direction and tone of the com¬ 
munity, but serve as mentors and provide personal support for community members. As a result, leaders can provide 
guidance and even consult on departmental and institutional changes being sought by community members, thereby 
creating a sense of personal support that is both localized in mentorship relationships and also pervasive across many 
experiences and activities available through the CoT. The combined value reported to arise from these key leaders and 
community culture suggests that these factors should be key considerations for CoTs to influence broader change in 
STEM reform. Specifically, these types of communities will gain by being intentional about the ways that their lead¬ 
ers represent and model cultures of support across a range of engagements and activities. 


Summary 


In general, we identified several key strategies for STEM reformers to use in order to engage faculty to contribute 
to broader organizational change outcomes at their home institutions. Reformers interested in starting or designing 
CoTs to contribute to STEM reform should a) provide adequate support to keep faculty involved for longer periods 
of time; b) seek involvement from multiple individuals within single institutions in their communities; c) engage 
community members in activities and development to help them gain mastery over the material in order to com¬ 
municate their work beyond the community; and d) identify key leaders who can both support faculty and foster 
a culture of engagement. By using these strategies, we found that these efforts can effectively engage faculty in the 
kinds of bottom-up change efforts advocated for in the literature and shown to correlate with broader institutional 
and departmental outcomes for STEM reform. In section 7, we shift attention to the broader shape of these commu¬ 
nities over time; in particular, we describe their lifecycles in order to highlight how effective communities form and 
continue to evolve. 
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VII. Formation and Lifecycle of 
C o m m u n itie s o f Tra n sfo rm a tio n 


O ur research found that the formation and lifecycle of these communities of transformation (CoTs) 
closely mirrors the literature on communities of practice (CoPs; reviewed in section 2). In this 
section, we describe the defining features of each lifecycle phase of a CoT: from potential to coalesc¬ 
ing, , followed by maturing and stewardship , and finally transformation. The maturing and stewardship 
phases entailed such important and distinctive characteristics from the traditional CoP literature 
that we go into greater depth on them in sections 9 and 10, respectively. 

The trajectories of these four CoTs were all similar in that they tended to start with developing the philosophy and 
then materials to support that philosophy They next moved to dissemination, and, lastly, they thought about com¬ 
munity development. Compared to traditional CoPs, the CoTs we studied spent much more time on domain and 
practice development before focusing on community. Yet, the leaders of each CoT acknowledged that thinking about 
the community development earlier would have been helpful. 


Potential Phase 


It is important to note that there can be some overlap in the first two phases—potential and coalescing—as com¬ 
munities are beginning to form. We present different events in these two phases, acknowledging that some of these 
events may occur in an earlier or later phase depending on how the community is forming. What is clear is that the 
events in the potential and coalescing phases do fall before the maturing phase. 

The four CoTs shared several characteristics in terms of aspects that defined their potential phase, including years 
of gestation, initial grants, key leaders coming together, efforts to develop and refine the philosophy, and the task of 
identifying a home. 

YEARS OF GES1AHON 

Each CoT spent several years refining its pedagogical approach and ideas for STEM reform, eventually resulting 
in the detailed philosophies each community currently espouses for improving STEM education. During the early 
years, courses and activities were piloted and tested. Faculty gathered to refine ideas about the STEM reform to be 
pursued. In retrospect, what seems critical for sustained CoTs is that many years be devoted to refining ideas; this 
is pivotal to long-term success. For example, SENCER started in 1989 with an interdisciplinary science course on 
AIDS at Rutgers University, but the course and its accompanying pedagogies were refined for over a decade before 
the leaders obtained funding and launched SENCER as a community. These early leaders gathered scientists at 
Rutgers who were concerned about whether conventional science education was adequate to the task of preparing 
students to solve real world problems. Similarly, the POGIL Project worked over the course of eight years to refine 
pedagogical ideas that connected guided, process-oriented approaches to inquiry. PKAL gathered for six to eight 
years to define their approach toward “what works” in undergraduate education. BioQUEST gathered computer 
scientists, philosophers, and scientists to develop ideas of the way computers could revolutionize teaching and make 
it more student-centered and engaging. 

Grants prior to becoming a formal community. One important way that the communities were able to test their 
potential was through grants they received to refine ideas. While the work started informally with faculty meeting at 
disciplinary conferences or within an institution, obtaining grants allowed them the time and resources to work to¬ 
gether to refine ideas, legitimating their work to a larger community. Additionally, grants allowed for collecting data 


Formation and Lifecycle of Communities of Transformation | 45 





about the efficacy of the ideas as they were put into practice. While all early-stage CoTs obtained grants, some ideas 
required even more support, due to the nature of the innovations they entailed. For example, BioQUEST computer 
simulations required more infrastructure support to develop initial models, and the community founders accordingly 
sought corporate and foundation funding for this. 

KEY LEADERS CO MING TOGETHER 

Each CoT was developed by an individual leader or a few individual leaders who had a passion for the ideas. These 
individuals provided the energy, passion, and resources to articulate the potential of these ideas and help move the 
communities toward the coalescing phase. It was essential to the success of the potential phase that there was a per¬ 
son who took up the mantle to support these ideas and pursue grants, working to gather people, to identify a home 
for the efforts, and to create conversations. While leadership continues to be important throughout the lifecycle, it 
is likely that none of these communities would have been able to persist beyond the potential phase without a key 
leader (or set of leaders) who drove the work of the fledgling community. Every person we interviewed underscored 
the importance of Jeanne Narum (PKAL), Rick Moog (the POGIL Project), David Burns (SENCER), and John 
Jungck (BioQUEST). Without these individuals, it is clear that the potential of the early ideas for reform would not 
have been propelled forward into the formation of these four communities of transformation. 

Additionally, prior to launching as a community, the organizers of these communities needed to draw in some key 
thought leaders in the sciences to provide legitimacy and foster involvement of others. In the earlier years, the com¬ 
munities hosted events that drew on the presence of such key leaders to ensure attendance by others. Several com¬ 
munities formed informal advisory boards as a strategy to bring in key leaders. For example, PKAL brought together 
academic leaders in liberal arts colleges to document evidence of “what works” in undergraduate STEM education. 
The POGIL Project gathered chemistry innovators in the mid-Atlantic area on a regular basis to discuss challenges 
in teaching chemistry. BioQUEST gathered computer scientists, STEM faculty, and philosophers to discuss the 
potential of technology to alter teaching in science. SENCER worked with deans through its connection with the 
Association of American Colleges and Universities (AAC&U), which helped to attract faculty to events. 

A HOME 

Each community of transformation was not part of an institution, but each found a home that provided a place for 
them to begin work and to locate their initial grants. While these homes did not necessarily provide resources or 
support, they did allow them a setting in which to work to centralize the efforts. The POGIL Project found a home 
with Franklin and Marshall College; SENCER began at Rutgers University before locating its first grant through 
AAC&U; BioQUEST began at Beloit College; and PKAL was located at the Independent Colleges Office. These 
institutional homes provided centralized hubs for information in terms of communication, website, and location for 
grants. They provided an important beginning place for the CoTs, but over time most relocated to more supportive 
environments as their goals became better refined. 

In summary, each of these CoTs share a common history of years spent in the early stages, refining their STEM re¬ 
form ideas, obtaining grants, finding key driving leaders, bringing in a set of influential faculty, and finding an initial 
home. These were essential years of gestation that carried each group through the stage of developing their potential, 
until they were prepared to launch more formally as full-fledged communities. This pre-work ensured that the first 
workshops and events held by each CoT were well attended by faculty, allowing the community to begin to coalesce. 


Coalescing 


As the communities of transformation entered the coalescing phase, they pursued several streams of essential work. 
Communities in this phase work to decisively name the STEM education problem, to connect themselves to the 
broader STEM reform movement, to develop key philosophical documents to distill the philosophies they had 
formulated, to foster the formation of their culture, to create key signature events, and to develop meaningful, useful 
materials. 
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NAMING THE STEM EDUCATION PROBLEM 

In preparation for effectively coalescing faculty, these CoTs were successful in identifying and naming a significant 
problem in STEM education (e.g., lack of relevance or contextualization, passive teaching, lack of student-centered 
approaches) and then offering a new approach to teaching that squarely addressed the named problem. For example, 
PKAL responded to the urgent need expressed in national reports to address the lack of STEM graduates and the 
high dropout rates in STEM. PKAL tied this problem to tendency of STEM faculty to use passive approaches to 
teaching, rather than the best evidence-based approaches. Similarly, SENCER articulated how STEM education was 
not preparing students for their responsibilities as citizens, including the responsibility to engage civic problems once 
the students graduated with their STEM degrees. The project therefore aimed, from its earliest stages, at making 
STEM education focus on key civic problems, like sustainability. By naming a problem and a needed reform, each of 
these CoTs created a movement and began to draw members toward key areas of interest that could be mobilized to 
address the task of reform. 

CONNECTION TO BROADER STEM REFORM MOVEMENT 

In addition to naming the work, these CoTs also connected their work to a broader STEM reform community 
Members of each CoT talked about interacting with faculty in other groups engaged in this work—members of Bio- 
QUEST were also part of PKAL, for example. The communities also organized gatherings with others who shared 
an interest in improving STEM education, both at disciplinary societies, and at the meetings of national organiza¬ 
tions and other interested groups, such as the National Academies of Sciences, the Howard Hughes Medical Insti¬ 
tute, and the National Science Foundation. PKAL worked with various STEM reform leaders to develop the ideas 
for Faculty for the 21st Century, which aimed to scale reform more broadly. These ideas and many others were drawn 
from the advice of national leaders in STEM reform. 


KEY BEG INNING DOCUMENTS 


Most of the CoTs solidified their philosophy into a key document in the coalescing phase as a way to formalize ideas 
that had been discussed during the potential phase. It was important to formalize the philosophy in this way in order 
to attract and recruit faculty to join the communities. Thus, the creation of key philosophical documents built upon 
the CoTs’ successes in naming the problems they faced. It worked synergistically with the communities’ new connec¬ 
tions to the broader STEM reform movement, allowing them to begin to gather dedicated memberships and become 
engaged communities. The key philosophical documents of each CoT are named below and briefly described. They 
are described in greater depth in Section 6. 


BioQUEST created the Three P’s: Problem-posing, 
problem-solving, and persuasion (Peterson &Jungck, 
1988). This approach involved more active engage¬ 
ment of students in the education process, through 
strategies such as having students teach one another 
and using curiosity and interest inspired by problem- 
posing to drive learning. 


The creation of key philosophical documents 
built upon the CoTs successes in naming the 
problems they faced. It worked synergistically 
with the communities' new connections to the 
broader STEM reform movement , allowing 
them to begin to gather dedicated membership, 
and become engaged communities. 


Project Kaleidoscope created a philosophy document called What Works: Building Natural Science Communities 
(PKAL, 1991). The document focused on active learning that is made relevant by being connected to real-world 
examples in context. This is achieved using techniques such as service learning, undergraduate research, and other 
engaged pedagogies, while taking into account students’ diverse backgrounds, thus making science a more collabora¬ 
tive and peer-based enterprise. 

SENCER created the SENCER Ideals (see Appendix 4A).The SENCER Ideals focus on capacious questions, con¬ 
text over content, interdisciplinary, and the connection of science to civic issues and relevant problems. These areas of 
pedagogical concern constituted a significant departure from science education at the time they were written. While 
the POGIL Project does not have a seminal or key document, their philosophy is summed up in the approach to 
pedagogy that is their namesake: process-oriented, guided-inquiry learning. 
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Each of these guiding philosophies for teaching had strong 
resonance with faculty, who reported to us the efficacy of 
writing down these philosophies so that they could be used 
to solidify ideas for STEM reform. In this way, these written 
documents helped to bring new people into the CoTs by 
energizing and activating them about the possibilities for 
change. 


The culture often reflected the 
philosophy that each community had 
developed, which became part of the 
core values of the group. 


C ULTURE FORMATION 

During this phase each group also developed a distinctive culture for which it became known. The culture often 
reflected the philosophy that each community had developed, which had become part of the core values of the group. 
These cultures also developed around core characteristics related to the leaders who had developed the CoTs, as well 
as the key and influential faculty who had joined over time. We describe the cultures of each of these communities of 
transformation below. 


PROJ ECTKALEIDOSCOPE. In interviews, participants identified the culture of PKAL as offering a safe space for 
experimentation and being supportive of faculty members’ needs, whether those needs be in pedagogical experimen¬ 
tation or career development. Respondents reiterated the roles of mentorship, relationships, and partnerships as un¬ 
derlying core characteristics of this community. As one faculty member noted: “People stay involved because of com¬ 
mitment to each other.” Faculty also spoke about the compelling and cutting-edge ideas at PKAL, the synthesis of 
many big ideas in science, the engaging sense of urgency for change, and the openness of the community toward new 
ideas and ways of thinking. Faculty commented on feeling empowered and challenged simultaneously by the culture 
of PKAL. They spoke about the energetic and friendly environment, and always about the passion around teaching. 

A focus and emphasis on leadership development was mentioned by most faculty, as well as a sense of accountability 
that comes with being a leader. Many described how PKAL embraced faculty across different disciplines, which was 
a strength of the community, since it created opportunities for cross-disciplinary conversations. Many people spoke 
about the PKAL way of conducting an event, which is one third introduction, one third active, small-group work, 
and one third reflection and accountability (i.e., what will you do once you leave?). PKAL events also include ritu¬ 
als that help members to demonstrate and reflect upon the community’s on-going commitment to one another. The 
PKAL practice of developing notebooks for events was also noted as an important resource for expressing cultural 
values, with inspirational quotes, places for reflection, and research and ideas to inform practice. 

THE PO GIL PROJECT. The POGIL Project’s culture was guided by the peer-to-peer model, with faculty learning 
from one another through workshops and other opportunities. Because of the emphasis on active learning and group 
work epitomized in their philosophy, the culture of the POGIL Project is built around small-group work and highly 
active interactions. Faculty spoke about the emphasis on feedback and assessment at the POGIL Project as very 
much a strong part of the culture, a practice that they really appreciated in order to support them as a research- or 
information-informed community. Participants noted a sense of intensity as part of the culture, which results from 
constant evaluation, questioning, and a driving curiosity. For some, this focus on evaluation and critique might lead to 
less of a sense of openness and inclusiveness. However, the POGIL Project also espouses a commitment to creativity, 
as exemplified by the practice of writing activities. Accordingly, many people describe their interest in being a part 
of the process of creating pedagogical materials. Many faculty also noted how the POGIL Project’s community was 
informed by a culture of science, in which it is okay to fail and to experiment, and learning takes place through trial 
and error. RickMoog, the POGIL Project’s leader, is noted for the slogan: “We work hard and have fun.’’Working 
hard is expressed through the intensity and sense of values that are parts of the culture, but the creativity, trial and 
error, and occasional light heartedness in social settings demonstrate the fun aspects of the community. According to 
our research, that ethic very much characterizes the culture of the POGIL Project. 

SENCER. Participants described SENCER’s culture as highly inclusive of all educators, whether they be students, 
faculty, staff, or informal educators. Community members described a minimal sense of hierarchy, exemplified by the 
fact that students were given equal voice to faculty. Those involved spoke about “all ideas being valuable and an open- 
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ness—this is non-elitist.’’There was also much discussion of feeling empowered, and of how this is connected to the 
lack of hierarchy and support. SENCER fosters a culture that is interdisciplinary, thereby supporting a view among 
its members that all sciences are important, and that there is no hierarchy related to one’s discipline. While we found 
this sense of inclusion among members, we also heard expressed a feeling of challenge, driven by a deep intellectual 
curiosity connected to the SENCER ideal of capacious questions. Faculty reiterated that they stay involved because 
of intellectual engagement that permeates the culture. The culture was described in terms of emotions, and many 
people spoke about SENCER helping them to regain their passion for teaching. SENCER’s culture connected them 
to the original reasons they started studying science. Faculty described how there was also a sense of meaning related 
to teaching and research, and that the goal of the community was to help people reconnect with a deeper sense of 
meaning about their work. They told us that teaching is not seen as a skill or strategy in the SENCER community, 
but as an art form and something that requires personal reflection. Creativity and departure from pre-packaged sets 
of ideas were seen as key elements within this community. This culture is expressed literally through the community’s 
insider language that are a part of each annual conference. 

HOQUEST. Like SENCER, BioQUEST was characterized by participants as a culture of “challenge.’’The commu¬ 
nity is designed to encourage faculty to move out of their safety zones. As a leader in the group noted: “This is not 
for people who like comfort; we are constantly rethinking teaching and exploring new areas of biology.’’There is a 
sense at BioQUEST of being on the cutting edge of biology, anticipating its future. The word “innovative” was used 
constantly to refer to the culture. Participants also noted BioQUEST as having a culture of creativity; the community 
does not provide pre-packaged ideas for teaching, but pushes people to develop their own approaches. In this way, 
teaching becomes a research activity, a scholarly inquiry. This dovetails with a belief that the faculty is just as much 
a learner as the student. The BioQUEST summer conference puts faculty in the role of student, and it uses active 
learning to have them explore an issue from that student perspective. Participants in the community used the meta¬ 
phor of a mobile teaching and learning center to describe their work. As in PKAL, many faculty in the BioQUEST 
community talked about their future-oriented culture as being constantly on the edge, developing new ideas that 
will work in the future. An important part of this orientation is the idea of openness—making all materials free and 
available. BioQUEST also has a very flat and nonhierarchical culture. They use the language of “coworkers,” no one 
is assumed to have more expertise than another, and there are no explicit leaders. Dissenting voices are encouraged, 
which participants note as “always welcome and [we] feel this is part of a healthy culture.” Most other CoTs were not 
characterized by this same sense of debate and active openness to disagreement. Over the years, BioQUEST par¬ 
ticipants could recount many different areas in which the leadership group itself had significant disagreements over 
direction. But they felt that the openness to debate helped them shape their direction productively and maintain a 
healthy community. They noted how, early on, the community was dominated more by men (an artifact of more men 
doing work in technology), but they described that this changed over time. With the shift to more women in the 
community, the culture also adopted more of a focus on diversity and social justice. 

DEVELOP A SIGNATURE EVENT 

Each group developed a key event that brought people together as a community around the philosophy. This became 
a regular event that defined the CoT in each case. For some communities, these initial signature events gave way 
to new signature events, while others maintained the same key events over their entire histories. For example, the 
POGIL Project developed its three-day workshops, in which faculty are introduced to the POGIL Project approach. 
These workshops bring together 20-30 faculty to learn from experienced the POGIL Project practitioners. Cam¬ 
puses typically hosted the workshops. The POGIL Project also started an annual meeting to bring together leaders 
in the movement in St. Louis; thus, it had continuity through maintaining the same setting each year. BioQUEST 
started a summer workshop—a small gathering of 25 faculty to explore the Three P’s philosophy. This hands-on work¬ 
shop has now been offered for over 25 years. PKAL started its assemblies that brought together faculty who wanted 
to support curriculum development around the What Works ideas, bringing together close to a hundred people for 
each assembly. Later, the leaders developed the Summer Leadership Institute as part of the Faculty for the 21st Cen¬ 
tury program (F21). This summer institute has remained constant for the last 20 years, even after the F21 program 
ended. SENCER offers an annual meeting that brings together several hundred people interested in the SENCER 
Ideals. This annual event has become SENCER’s signature event, and it attracts substantial repeat attendance. 
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DEVELOP MEANING FUL MATERIALS 

In addition to the core philosophy document, each organization developed a series of resources and materials that 
were supportive of the educational reform efforts that they promote. For example, SENCER developed course 
modules for what a SENCER-ized class looks like, so that individuals had samples for creating their own syllabi. 
They also created a journal to enable people to publish information about their revised curricula and pedagogical ap¬ 
proaches. They developed assessment materials to help individuals in evaluating their new SENCER courses. Bio- 
QUEST created a library of course materials (e.g., computer simulations) that was searchable and kept up-to-date 
for about 20 years. They have now transitioned to a website with curricular resources, particularly case studies, ways 
to use Excel for assignments, and other simulation-related materials that require less of a technology infrastructure 
to use. They made this transition to help less resource-rich institutions to integrate the Three P’s philosophy. PKAL 
was known for creating reports after their assemblies to capture key ideas related to STEM reform. For example, 
they had an assembly on revising introductory science courses, and the ensuing report became a resource for ongoing 
change. The POGIL Project developed curricular activities and branded them as having been reviewed and approved 
by the POGIL Project professionals. In addition, they created textbooks that included the POGIL Project activities. 
Each of their workshops is accompanied by carefully developed handouts focused on helping individuals create the 
POGIL Project materials for themselves. 

Thus, each community developed a set of important resources that became invaluable to their members at multiple 
tiers of engagement: introducing the STEM reform, practicing the reform, and advancing the work on both the 
individual and institutional levels. For example, the POGIL Project workshop materials were introductory resources 
for educators, while the textbooks and branded activities were intended to help more mid-level professionals further 
their practice. Finally, for POGIL, the creation of new materials was often a task for more advanced practitioners. 


Maturing 


Expansion was a major activity during the maturing phase for each of these communities of transformation, and we 
have dedicated Section 8 to discussing their expansion strategies. However, in this section we describe several other 
important activities that happened during the maturing phase, including creating a rhythm of the events for different 
stages on involvement, obtaining new grants, building community, leadership development, and the emergence of a 
distributed leadership. The maturing phase is characterized by considering ways to manage the growth of the enlarg¬ 
ing community while also continuing and expanding that growth. 

C REATE A RHYTHM OF THE EVENTS FOR DIFFERENT STAG ES OF INVOLVEMENT 

Each of the CoTs recognized that, to develop the community, they would need to have a rhythm of events that 
helped people move from introductory relationships with the practice to more advanced work, in which faculty could 
play an increasing leadership and mentorship role. The POGIL Project first created an introductory workshop, and 
then they later recognized the need for a more advanced workshop for individuals who wanted to follow up and 
learn more about creating activities. Those who showed interest in greater leadership involvement were encouraged to 
begin facilitating workshops, which often led them to be invited to participate in the annual national meeting, where 
a small leadership group gathered. SENCER had a similar rhythm of events, beginning with individuals becoming 
introduced to the community through attending their annual conference. In later years, individuals would be invited 
to be speakers at the summer conference. As faculty became more involved, they were often invited to the Washing¬ 
ton Symposium or became leadership fellows. PKAL started off by inviting people to assemblies or to the Leader¬ 
ship Institute as an introduction. Next, engaged participants were invited to be speakers at these events, and then, 
later, they were more formally designated as mentors. These advanced mentors were connected to newer people who 
showed interest in becoming more involved in the community. PKAL also has mechanisms for identifying people 
who want to become more involved and increase their participation. The POGIL Project had workshop leaders des¬ 
ignate individuals who showed particular interest, and those participants would receive follow-up contact from The 
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POGIL Project leaders at the home office. 


NEW GRANTS 

In order to develop new materials—a need that continues from the coalescing into the maturing phase—and later to 
expand to new audiences, each CoT developed and obtained new grants. For example, when BioQUEST wanted to 
reach community colleges, they created and gained funding for the C3 Cyberlearning Project. Similarly, CASE IT! 
was a grant to create case study activities for molecular biology and to expand reach into less resource-rich institu¬ 
tions. BEDROCK was a grant-funded initiative to create resources related to bioinformatics to get more mathemat¬ 
ics into biology courses. These are examples of how their grants and the resultant materials helped them to expand 
into new sectors, like the community college, or into different courses, such as molecular biology. SENCER was 
interested in working with informal educators from museums and libraries, so it developed a grant to bring these in¬ 
dividuals to their events and to connect them more directly to science faculty within the community. PKAL obtained 
a grant from the National Science Foundation to expand its regional networks so they could support more faculty 
in institutions across the country. PKAL also obtained funding to establish a subgroup, Faculty for Undergraduate 
Neuroscience, that would bring together faculty in this area to learn about more engaged pedagogy. Thus, new grants 
helped PKAL expand and extend its What Works principles by reaching into a new discipline. In this way, PKAL 
successfully moved to new types of institutions and across different fields of science, using grants to gather key 
educators in these new arenas and to develop customized materials to help keep its pedagogical approach appropri¬ 
ate to this new context. Similarly, the POGIL Project began work with high school teachers and discovered that they 
wanted access to more pre-packaged materials, rather than to create them independently. A grant helped the POGIL 
Project to create more pre-packaged activities for these teachers, thus extending the impact of the pedagogical inno¬ 
vation. These examples show how, while grants are critical to growth, they are also pivotal to maintaining the com¬ 
munities and their core functions. We will see how grants continue to serve a role in the stewardship phase, discussed 
in section 10. 

BUILDING COMMUNITY 

Each community of transformation had several strategies for ensuring or developing community over time. Bio- 
QUEST used the strategy of inviting one third new individuals, one third returning individuals, and one third more 
senior individuals to serve as leaders at each summer institute. Their goal was to have an intergenerational approach, 
in which faculty who are the new individuals attending the event would be able to interact with more senior people 
who are returning to the event. SENCER similarly strove for an intergenerational approach by inviting newcomers 
as well as returning individuals to their summer conference. They encouraged delegations from institutions to include 
individuals at different career stages when attending the annual conference and applying for their mini-grant pro¬ 
gram. 

Another strategy for building community is to have two-way communication. Too often, intentional communities 
can be hampered by one-way communication, where individuals in the community receive communications, but they 
have few mechanisms for communicating with the leadership. Over time, each community we studied developed 
mechanisms for obtaining feedback, ranging from surveys, time at the end of events for open dialogue, and follow¬ 
up after attendance at events. For example, PKAL has a tradition of ending each event with people sharing how the 
experience has had an impact on them and allowing them to provide feedback to improve the experience for future 
participants. The POGIL Project has a feedback form that they provide at every event called the Strengths, Improve¬ 
ment, and Insights (SII) tool. The SII garners feedback to improve all of their events and to better understand the 
changing needs of the people in their community. 

Another major strategy for building community in each of these CoTs was to invite individual participation in vari¬ 
ous activities. SENCER, for example, invites individuals to present at the summer conference, to submit mini-grants, 
to apply to be leadership fellows, to submit to the SENCER journal, and to create course modules for the library. 
BioQUEST reaches out to individuals to publish articles based on their work at the Summer Workshop, to partici¬ 
pate in submission of new grants to develop materials, to work on creating new teaching tools, and to present at vari¬ 
ous conferences about new teaching materials. They do not wait passively for individuals to submit materials 
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or provide ideas for possible grants; rather, they actively reach out to individuals on an ongoing basis to invite their 
involvement in these activities, thus building engagement with the overall community. 

LEADERSHIP DEVELOPMENT 

Over time, as the communities developed, there was recognition that new faculty wanted to be mentored and sup¬ 
ported in conducting this work. Simultaneously, the CoTs recognized that they needed faculty to lead events, to cre¬ 
ate materials, and to develop new grants. As a result of this dual awareness, these communities came to recognize the 
importance of developing leaders. Leadership development is linked to the rhythm of events for different levels of 
participation, as we discussed above. All the more advanced modes of participation—such as presenting at workshops 
or conferences, working on grants, serving as mentors, being asked to publish, gaining opportunities to be visiting 
scholars, serving on advisory boards and committees, and receiving recognitions such as fellowships—are venues for 
developing leaders and sustaining them within the communities. Many of the faculty we spoke with said that they 
would have been less likely to continue participation if there had not been ways for them to give back to the com¬ 
munity. Leaders in these CoTs recognized the need to create opportunities for participants to play leadership roles 
to maintain the engagement of individuals who had contributed to advancing ideas and practices. Additionally, these 
advanced faculty members expressed interest in giving back to their communities. The CoTs responded by providing 
a variety of opportunities for this next level of engagement. 

Most leadership development we encountered in these communities was informal, offered through opportunities to 
engage in presentations, mentoring, and grant activities. However some of the CoTs also have created much more 
formal leadership development activities. For example, PKAL’s Summer Leadership Institute was aimed at foster¬ 
ing the skills of individuals who could go on to create institutional and departmental changes. The skills participants 
acquired in workshops such as these were also critical for their growth as leaders within the CoTs. 

DISTRIBUTED LEADERSHIP 

The CoTs in the study recognized that distributing leadership was essential for the growth and health of the commu¬ 
nity. To distribute leadership meant that a large cadre of faculty participated in leadership development activities, en¬ 
abling them to become key contributors in planning events, creating publications and materials, serving as mentors, 
and playing the many important roles necessary for these communities to grow and thrive. While each community 
began with a small group of dedicated leaders, as the communities grew during the maturing phase they recognized 
that there would be no way to serve their members well without delegating responsibility. This was the impetus for 
these leadership development activities described above. 

For example, PKAL developed a group known as the village elders that provided mentoring to various faculty, served 
on advisory boards, and presented at meetings and events. This group of individuals grew over time to perform all 
of these beneficial activities and roles within the community. In turn, SENCER fostered leadership by setting up 
regional innovation centers and by training and developing leaders for each of the centers. They also established 
regional networks and provided leadership training to support individuals working to grow these networks. POGIL 
used its annual national meeting to bring together a large group of rising leaders, and the community worked to 
ensure that leadership was distributed among this group. With this broad base of leadership, POGIL continue to 
develop materials, create new grants, reach out to new groups, and strategize about the future of the community. 
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Stewardship 


Stewardship is about making the community sustainable over time. This was such a larger area of our findings that 
we devote section 10 entirely to the concepts and recommendations that emerged. The focus on sustainability devel¬ 
oped organically for these communities, as their growing impact made clear the need to secure the continuity of their 
work. For these four mature CoTs, sustainability was central to the task of meeting goals and increasing impact, as we 
discuss in that section. 


Transformation 


As noted in the literature review on communities of practice (see section 2), transformation is the stage when a com¬ 
munity faces a crisis that forces it to reexamine its purpose and whether it should continue. Two of the communities 
in our study, PKAL and BioQUEST, underwent transformations, and they have continued to grow dynamically after 
that process. We will describe their stories. 

PKAL’s longtime leader retired, and the community needed to find a new home for its work; it eventually became a 
part of AAC&.U. The advisory board for the community underwent a process to examine whether the organization 
still served a purpose and should continue. In response to the retirement of the longtime leader, who had provided 
so much support, and the known difficulties ahead of finding new leadership and a permanent home for PKAL, the 
advisory board went on a long soul-searching process to examine the degree to which PKAL still had an important 
and distinct role to play in STEM reform. While they recognized that many different groups were now focused on 
pedagogical innovation, they identified that there were no other communities focused on leadership development in 
STEM. They also observed that PKAL’s strong focus on diversity was not embraced or championed by other groups. 
Through an examination of their work, the advisory board identified that there was an important need to continue 
the work of PKAL, and they ended up embracing their transitioning to a new leader and a new home. 

BioQUEST underwent two successful transitions of leadership, but its primary transition was precipitated by fi¬ 
nancial uncertainty. The tight budget times and difficulty obtaining grants in recent years forced the community to 
examine its purpose and to determine whether it should become a nonprofit, and whether it had enough support to 
continue. Over the history of this community, there have been several transitions of content and priorities, including 
the shift from focusing on computer simulations to using other forms of technologies to create engaging pedago¬ 
gies and curricula. When BioQUEST began, there were almost no other efforts focused on this type of pedagogical 
approach. Through the reflection necessitated by the financial climate, BioQUEST assessed the landscape and found 
that their approach remained unique and viable; as a result, they have created a nonprofit organization to continue 
supporting the community. 


Summary 


This section described how these communities of transformation generally follow similar trajectories, as they recog¬ 
nize their potential for approaching a STEM reform problem, coalesce around the issue and begin to form community, 
mature into full-fledged communities with expanding impact, and steward the community onward. In response to the 
inevitable demands of change over time, communities are eventually forced to reflect and transform accordingly. 

Some of our findings related to the maturing and stewardship phases provide useful information for future commu¬ 
nities. We describe these findings in more detail in sections 9 and 10. First, we turn in section 8 to common chal¬ 
lenges these communities encountered over the course of their evolution. 
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VIII. Common Challenges in Evolution 


O ver the course of their lifecycles, not only did these communities of transformation (CoTs) follow 
certain common strategies and activities, but they also experienced some common challenges. These 
challenges could not be charted to fit neatly into particular stages of the lifecycle. For example, fund¬ 
ing was a problem for some in th t potential phase, but it also influenced communities when they 
were maturing , and again played a role in the transformation stage. Similarly, the challenge posed by 
a shift in content focus can occur as a result of the expansion of membership that takes place in the maturing phase, 
but it might also occur later, during stewardship as well. We identified from our research 10 key challenges experi¬ 
enced by these communities that are strongly related to the sustainability model we will describe in section 10. These 
challenges are: funding, shifting focus, leadership becoming too identified with an individual leader, the tension be¬ 
tween focusing on the community and focusing on projects, staleness, seeking legitimacy, counteracting the dominant 
culture of science education, maintaining personal and community integrity, the tension between focusing on general 
faculty development and focusing on a specific technique, and increasing demands on faculty. Where we observed 
that the communities developed solutions to these challenges, we highlight those strategies in italics to guide future 
communities. 


Funding 


In general, funding obstacles required strategic trade-offs and created challenges related to having enough staff, be¬ 
ing able to create needed materials, and being able to support community members in meaningful ways, such as by 
hosting quality events to help them to accomplish their goals. All four CoTs encountered funding challenges at vari¬ 
ous times in their lifecycles. In response, some networks altered or expanded their domains of knowledge to pursue 
funding opportunities. For instance, one SENCER participant stated that the community was only able to survive 
because of its ability to branch out to different areas of instruction—general education areas, graduate courses, new 
chemistry courses, etc. SENCER branched out in this way “because all of these different areas provide new initiatives 
for funding.” 

Funding challenges often affected the community indirectly, through impact on network events and meetings that 
needed to be cancelled, downsized, or held in inconvenient locations. Some community members were directly af¬ 
fected through the elimination of travel grants and reductions in salary, and a few project leaders continued to work 
without receiving salaries. According to one participant, PKAL’s principal investigator never received a salary, but 
would actively seek out funds for the staff. One CoT leader explained that, when “you care so much about one an¬ 
other’s lives,” the struggle to keep things going when salaries have been cut is particularly difficult. 

Funding struggles tended to impact CoTs the most by limiting their ability to hire and retain administrative staff, 
to recruit new members, to develop new leadership, and to train community members to develop materials. Despite 
their grassroots beginnings, all four CoTs engaged in recruitment efforts to disseminate their STEM reform mis¬ 
sions to larger audiences (described in section 9). These expansion efforts were much easier to sustain in the early 
days, when the pool of participants was small, but when the communities began to grow, grant money became insuf¬ 
ficient—not enough to support experience that would foster strong ties or develop and disseminate best practices. 

The CoTs took a variety of approaches to solving funding challenges, but the common theme that came through in 
interviews was that it is important to maintain flexibility in goal setting, and openness to new ideas andfunding opportuni¬ 
ties. Uncertain financial futures sometimes prevented the CoTs from becoming over-committed to their reform goals. 
Though each CoT began with a particular articulation of the need for STEM reform and a mission to contribute 
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to that task, they have each had to evolve and adapt over time in order to secure funding and persist. Adaptability is 
one feature that has contributed to the success of these CoTs, and the ability to adapt without losing touch with that 
original mission has been crucial to their sustained impact. 

It is important to note that many of the other challenges described below stem from issues related to funding. In 
pursuit of funding, communities can shift purpose, become too project- rather than community-oriented, and try to 
expand to new groups/institutions/disciplines for which they do not have adequate support. It is important for orga¬ 
nizations that support these communities to understand these dilemmas that arise from funding shortages. Funders 
and other support organizations can alleviate many of these challenges by offering strategies for navigating funding 
obstacles, such as bridge funding, on-going support through a sustained financial model (see sustainability section of 
this report for examples), or partnerships with other funding organizations to continue support for CoTs that dem¬ 
onstrate promise. Our study uncovered “piecemeal” strategies that CoTs utilized for navigating funding challenges, 
but we recommend that CoTs in partnership with funders develop more systematic solutions for how their commu¬ 
nities can be continuously supported and maintained. 


Shifting Focus 


Related to the need for flexibility, some of the CoTs encountered obstacles when attempting to shift the focus of 
their work. Shifting focus refers to the intentional adaptations that these communities undertake in response to 
changes in the larger environment surrounding STEM reform. These environmental inputs included financial con¬ 
straints or interests of potential funders, needs of the community, and needs of students. For example, the POGIL 
Project decided to commercialize their materials as a solution to unpredictable funding mechanisms. To do this, the 
community shifted its focus from college chemistry to high school science classes, and community leaders developed 
a curriculum that they could market to school districts, thereby generating sufficient revenue to sustain themselves. 
Problems arose for the POGIL Project because high school curricula did not conform to the principles endorsed 
in the original mission, and this area of work “wasn’t specifically written into the original grant funding [the com¬ 
munity].” One leader explained that “there was this desire to be faithful to not only the grant and the funding, but 
to the original idea’The tension arose between this sense of loyalty and the need to continue to grow. For the com¬ 
munities that attempted to shift focus and expand to new membership and new domains, a primary challenge arose 
as they tried to find the leadership and expertise necessary to steer them into these new territories, to draft materials 

for these subjects, and to attract diverse, new commu¬ 
nity members. This challenge highlighted the tension 
between pursuing viable, secure avenues for the com¬ 
munity and being opportunistic about new possibilities. 
At times a CoT made the mistake of following a new 
area or expanding to a new group simply because the 
opportunity existed. The pursuit of such a new direc¬ 
tion without consideration of the community’s over¬ 
arching mission led to turmoil and challenge. 

Communities that encountered these challenges overcame 
them by reflecting on their purpose, and by deciding how best they could serve their community with limited options. The 
decisions were often difficult, requiring the CoTs to decide between the immediate needs of community members 
and the needs of the community or mission itself. It will become apparent in the review of the sustainability model 
in section 10 that these challenges could only be navigated successfully through obtaining on-going feedback and as¬ 
sessing work. These were ways for community leaders to reflect on their practice and to ensure that they shifted when 
needed, but also maintained the essential focus. Another strategy for a successful shift in focus was to communicate clearly 
with the community about the rationale behind a possible change in direction. For example, BioQUEST let faculty know 
that its shift from curricular development to faculty development was in response to a need to disseminate ideas more 
broadly. 


Funders and other support organizations can 
alleviate many of these challenges by offering 
strategies for navigating funding obstacles, 
such as bridge funding, on-going through a 
sustained financial model, or partnerships 
with other funding organizations to continue 
support for CoTs that demonstrate promise. 
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Community Leadership Too Much Identified with an Individual Leader 


Each of these communities was started by a charismatic individual, who became strongly identified with the com¬ 
munity as its driving force. Over time, though, each community realized that it needed to distribute leadership, a step 
that became critical to sustaining the community in the maturing and stewardship phases. However, each community 
experienced a challenge associated with shifting authority, decisions, and identification to a broader set of individuals. 
The role of voice or spokesperson for the community was often narrowly embedded in a single leader or two. Various 
faculty members commented: “When you think about PKAL, you think of Jeanne Narum”; or “SENCER would not 
be SENCER without David Burns”; or “John Jungckwas a really influential leader and father of BioQUEST”; or 
“RickMoog—he lives POGIL; he is what people identify with.” 

The leadership challenge was to maintain the strong, charismatic leadership that inspired people to join and engaged 
current members, but also to distribute the leadership so that in a time of leadership transition, or health leave, or 
another situation in which the main leaders could not meet the needs of the community, there were others who could 
effectively do the work. For the POGIL Project, the annual steering committee became a venue for training leaders 
and distributing leadership to a broader group. When we observed this meeting, we noticed how core the POGIL 
Project leadership was spread over seven or eight faculty members, with each one equipped to make key decisions 
and also to speak as the voice of the POGIL Project. These individuals led strategic planning efforts, gave regular 
presentations, and headed major grant projects. Another example of distribution of leadership was BioQUEST’s ef¬ 
fort to create a leadership team as John Jungck stepped down, rather than to just replace him with a single individual 
as leader. Similar to the POGIL Project, a subset of individuals began planning events, giving presentations, and 
defining strategic directions for BioQUEST. Thus, the key strategy for overcoming the challenge of distributing leadership 
is to create a structure that locates leadership within a team, steering committee, or planning group, thereby expanding leader¬ 
ship and creating infrastructure forfurther leadership development. It is worth noting that advisory boards did not appear 
to offer enough of a structure to achieve this purpose, as they still often relied on an individual leader who remained 
the seat of decision-making. 


Project-Foe used versus Community-Foe used Decisions 


The urge to pursue particular opportunities—often because of funding—can result in a community becoming proj¬ 
ect- rather than community-focused. Often a grant-funded project focuses on a particular idea—interdisciplinary 
teaching, involving informal educators, integrating more math into teaching a science discipline, a geographically- 
oriented service project, etc. The reason for the narrow project focus is that funders desire new areas to be explored 
and are reluctant to continue funding a good idea to be disseminated and continued. However, even when such 
funding streams were directed at tangential projects, they usually allowed the community some infrastructure money 
to help it to continue its core work. While these types of projects generally became incorporated into the broader 
community ideas for improved practice sooner or later, the faculty described them as projects that they either “did not 
see as central to the community” or “were waiting to see the purpose. "This led the leaders of the CoTs to spend a sig¬ 
nificant amount of their time and energy on the task of shepherding such project through the community, as opposed 
to working on fostering the overall community. Accordingly, there was a concern expressed often among faculty 
that the community received less attention when various projects mounted. Faculty were typically interested in the 
philosophical underpinnings of the community and its space for improving practice, not in its particular projects or 
initiatives. Yet, the communities needed to secure funding in order to support the infrastructure to hold their annual 
events; grant-funded projects provided the way to make these community events happen. One CoT leader described 
this challenge: “You can’t get funding for helping faculty improve their practice in general. So we have to write these 
very specific project proposals. We know that this has us focus then on these projects, but there is no other way to 
stay afloat. We tried writing more general grants, but they never get funded.” Compare this perspective to the way a 
faculty member describes the feeling as leaders pursue their grant projects: “I worry about the community; they don’t 
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see themselves in some of these new projects. I see less and less people continuing their involvement. This will be¬ 
come more of an organization that people cycle through rather than an on-going community.” The communities in this 
study addressed this challenge by ensuring that some projectfunding went to more general supportfor the community. They 
also worked to translate grant ideas into more general lessons or ideas that would have resonance with the larger community. 


Sta leness 


All of the communities dealt, to some extent, with the sense of staleness that arose after the first wave of enthusiasm 
in the work faded. Across its ideas, leadership, and written materials, as a community widens its boundaries it risks 
diluting its intensity, as “changing markets, organizational structures, and technology can render the community’s 
domain irrelevant” (Wenger et al., 2002, p. 109). For instance, one SENCER participant complained of attending 
the same lecture at multiple events: “I feel like they have evolved, but in some ways maybe not as much as Ed like to 
see, because it seems like we’re still relying on the same [ideas].. .what was great in 1990 [is] not the best thing since 
sliced bread in 1998.” One BioQUEST participant stated that “new people are not as powerful a force as you might 
think because the collective group that is committed to volunteerism has to also be committed to being self-critical” 
and to reflecting on “what’s not working or what’s working well, what’s changed as an external dynamic that you 
need to adjust to.” Staleness can prevent communities from evolving. Interview participants suggested that successful 
communities must remind themselves that they “don’t know everything,” and these communities should “never get to 
the point where we think that [they] know exactly how things should be and how people should do things.” 

Solutions to staleness depended on the particular cause of the problem. For some communities , stale leadership was a 
barrier to growth , so new leaders and new voices were essential to community survival. One leader stated, “the biggest 
challenge is to make sure that new leadership comes in,” but it is important that leadership does not stray too far 
from the original vision of the community, because this could turn away long-standing community members. For 
some, the primary concern was in fact keeping their mission and philosophy intact. As a result, these communities dealt 
with staleness by using technology and new ideas to develop new tools, thus changing their approach while retaining their 
philosophy and identity as a community. Note, however, that solutions to staleness can also lead to other challenges in 
the future, such as balancing grants in new project directions with the ongoing needs of the existing community, as 
discussed above. 


Legitimacy 


Legitimacy is best described as recognition by dominant institutional structures. The dominant structures that define 
legitimacy for these CoTs are the norms and values that govern the mainstream academic community. Due to a 
multitude of factors, all four CoTs to some degree lack legitimacy in this sense. One of the primary reasons for this is 
their focus on teaching and pedagogy in the higher education community. As most CoT members at the time of this 
research were faculty members employed at post-secondary institutions, participants often felt pressures from their 
colleagues and administrations to focus on different aspects of faculty life, such as research and grant funding. Faculty 
members who concern themselves primarily with teaching tend to exist on the periphery of dominant structures in 
academia. For instance, one SENCER member described the community as comprised of “fringe faculty.” SENCER 
members were “the ones who were not avoiding the complex issues, like climate change, or acid deposition, or habitat 
fragmentation, or equity issues. You know, so we started out as a fringe faculty, but we believed in what we were do¬ 
ing.” One POGIL Project leader stated that “beginning chemistry instructors have told me that, if they were to try to 
do any of this POGIL stuff, they would not be given tenure.”The legitimacy challenge was closely intertwined with 
recruitment and engagement, as a lack of community legitimacy tended to deter potential members who sought ac¬ 
ceptance through dominant institutional structures, like research and publishing. 
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The communities tried to address this deficiency of legitimacy for their members by creating awards, providing fellowships, 
awarding seedfunding and grants, and offering other ways to recognize members of their community. They also gained legiti¬ 
macy by working with national organizations such as the National Academy of Sciences, and by recruiting influentialfaculty 
leaders. The communities additionally bolstered legitimacy for their approach to STEM reform by engaging in research to 
gather evidence of the effcacy of their work. 


The Dominant Culture of Science Education 


Members of each CoT identified a shared cultural challenge common across efforts to reform STEM pedagogy. 
Participants reported that the science community values content knowledge as the most important principle of edu¬ 
cation, so efforts to improve teaching were resisted if they threatened ideas about what material would be covered in 
class. This cultural aspect of the art of science pedagogy was noted as an issue that each community had to navigate 
in order for their community to engage faculty, to obtain buy-in, and to help faculty engaged in innovations to be 
successful on their home campuses. The communities addressed this challenge by questioning this assumption (andprovid¬ 
ing a rationale for why it was not valid) and equipping faculty with counter-narratives to help them argue in support of their 
approaches to teaching. 


Maintaining Community Integrity 


A common theme across the data was the struggle the communities faced between staying true to their original mis¬ 
sions and evolving or adapting to changes in their environments in order to survive and sustain themselves. The chal¬ 
lenge of maintaining integrity refers to the tension that arises when the orientation of the larger STEM community 
is not aligned with the needs of the particular CoT and its members. 

For some of the CoTs, these challenges manifested as matters of loyalty—some had difficulty sustaining the same 
kind of personal commitment from their members that they 
had inspired in the early stages. For instance, PKAL’s grassroots 
beginnings attracted a community of deeply devoted followers, 
but after PKAL’s merger with AAC&U, there were concerns 
among the community members that the people and traditions 
at the heart of PKAL would be absorbed by the larger organiza¬ 
tion. One participant stated, “Because it’s such an old [commu¬ 
nity] , and there are such deep loyalties, to move so quickly with 
changes and new initiatives would have been a mistake, I think, 
without really having a deep appreciation for those traditions and 
what they mean to the people who hold them so dear.” A simi¬ 
lar struggle emerged for the CoTs that had developed regional 
groups as a strategy to counteract the challenges of geographic 
distance. For example, SENCER’s regional centers were success¬ 
ful in some regions and not others, mainly because participants 
struggled to feel connected to the larger purpose within their 
separate centers. Community leaders described the difficulty of developing a cohesive identity that could translate 
from the national to regional groups; effective leadership of the regional centers largely depended on this identity. 
One SENCER participant reported, “The faces of the regional projects are not necessarily people who connect and 
foster community in quite the same way. And so I never felt a connection to the southern center, I never felt com¬ 
pelled to participate in stuff down there.” 


The most effective solution to the 
challenge of maintaining integrity 
is flexibility - the ability to adapt 
to uncertain environments. This 
often required self-reflection on the 
community's mission and philosophy, 
and conscious decision-making about 
which aspects of the community, if any, 
can be compromised in the interest of 
survival. 
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Other challenges of integrity tended to accompany the management of knowledge and materials in the face of 
growth of the CoTs. As all of the communities expanded from their grassroots origins, they had to evolve in order to 
remain relevant to the communities that they served; with continued growth, this became a more difficult task. Ac¬ 
cording to Wenger, McDermott, and Snyder (2002), “established communities regularly experience a tension between 
developing their own tools, methods, and approaches and being open to new ideas and members” (p. 104). New 
members were essential to ensure growth, and growth was necessary for survival. However, new members brought 
new ideas, and in order to benefit from these ideas, the community needed to display a certain degree of flexibility in 
both its structure and its knowledge domains. 

The most effective solution to the challenge of maintaining integrity appeared to beflexibility—the ability to adapt to uncer¬ 
tain environments. This often required self reflection on the community’s mission and philosophy, and conscious decision-mak¬ 
ing about which aspects of the community, if any, can be compromised in the interest of survival. For example, in order to 
maintain its unique identity and approach, the POGIL Project began to brand its materials. The community leaders 
determined what distinguished the POGIL Project activities from the non-POGIL Project activities, and they began 
to certify materials based on standardized requirements. Another example is PKAL, which was able to grow while 
maintaining existing ties by respecting long-standing traditions, and by asking that members acknowledge their new 
role “to explore new things for PKAL, new directions, expanded directions, but to do that while also honoring what 
PKAL has historically meant to those who have been involved.” 


Foe us on General Faculty Improvement versus a Specific Pedagogical 
Approach 


Each of the communities, with the exception of the POGIL Project, focuses broadly on using evidence-based teach¬ 
ing practices and on improving teaching, but not on single, specific techniques. Even the POGIL Project, which had 
a more particular approach, reported movement away from a “pure” POGIL Project approach toward the broader 
inclusion of evidence-based, active-learning strategies. The other three communities advocate for a broader philo¬ 
sophical re-orientation in STEM education: to think about teaching as a scholarly activity, to consider and reflect 
on teaching, and to review and inform one’s practice after considering a variety of approaches that may fit different 
styles, disciplines, or purposes. One faculty member commented on this challenge: “When I first was introduced to 
PKAL, I was confused: What were they advocating for? I was looking for a specific idea, and they offered many dif¬ 
ferent ideas. And, over time, new ideas kept being introduced. Not until I fully understood what was being asked of 
me—to consider and reflect on my teaching, not just to adopt a practice—did I appreciate or understand. I know that 
it is hard for other faculty to understand as well.” 

In order to address this challenge, the CoTs developed partnerships with members of the teaching and learning improve¬ 
ment community who provided a language and framework for their work. For example, the POGIL Project invited many 
faculty who conduct research on teaching, identified as discipline-based education researchers (DBER), to join and 
inform the community. These faculty attended the POGIL Project’s leadership events, including their national steer¬ 
ing committee meetings, and helped the community by communicating this broader approach to improving teaching 
practice. PKAL also involved DBER faculty and invited them as speakers at events. Similarly, BioQUEST partnered 
with individuals from the faculty-development community to help introduce a shared language and a network of 
people who could provide advice from the scholarship of teaching. Beyond these individual community efforts, the 
STEM community as a whole is becoming increasingly familiar with the scholarship of teaching, informed by recent 
reports from the National Academy of Sciences (2012) on the history of DBER, as well as by key findings shared 
by other high-profile organizations. This sector-wide shift provides a stronger foundation for the work of these first 
CoTs focused on general teaching improvement. Leaders in the CoTs were careful to note, however, that communi¬ 
ties need to bring in “mainline science”faculty to remain legitimate, notjust gather together a subset of faculty viewed by the 
disciplines as !'science educators.' 
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Increasing and Changing Demands on Faculty 


A growing challenge for these CoTs relates to the changes in faculty workforce. First, each community noted that 
faculty are increasingly employed in contingent positions (70% of faculty nationally) and that this trend is prevalent 
across all institutional types and most disciplines, particularly math, chemistry, and biology (for more details about 
these trends see www.thechangingfaculty.org). Contingent or non-tenure-track faculty are typically given no funds to 
support professional development, which makes their involvement in these communities problematic. Leaders within 
the CoTs noted their concerns about the future of STEM reform: “How will we continue this work when the major¬ 
ity of faculty do not have positions that allow them to participate as professionals and improve their practice?” Unless 
there are shifts in institutional policies and pressures from outside, faculty will not be able to benefit from these com¬ 
munities. 

Second, tenure-track faculty are pulled in many directions, are increasingly overloaded with institutional service and 
leadership, and are required to obtain more grants than they were in the past, making involvement in these com¬ 
munities challenging. As a leader in one of the CoTs commented: “It is harder and harder to get the attention of the 
up-and-coming generation; they have less time and I fear for the future of communities like ours, given the current 
trends in faculty hiring and the demands on young tenure-track faculty.” 

Third, in certain sectors, such as community colleges, faculty are stretched and overcommitted to the point that it is 
hard for them to find the time to improve their teaching. Heavy teaching loads and year-round teaching can make 
professional development a luxury that is out of reach. BioQUEST has been working with community colleges for 
nearly a decade, and it understands the struggle of faculty in this sector to become and remain involved. In general, 
however, while this is an important and evolving challenge, the CoTs have not intentionally addressed the changing 
needs and demands placed on the faculty today. 


Summary 


The communities of transformation in our study encountered many of the same challenges as they evolved and grew 
as communities. Understanding and acknowledging the challenges these communities face, and the strategies they 
use to respond, can help future communities as they encounter similar challenges. While many of these challenges 
can stem from funding issues, they each pose unique problems to be solved by growing communities; we have tried 
to highlight some of the most effective strategies above. In the following section, we describe the ways in which the 
CoTs in our study expanded their scope in the maturing stage by focusing on new areas, such as disciplines and edu¬ 
cational sectors, for pursuing STEM reform. 
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IX. Expansion Strategies: More on the 
Maturing Phase 


A s noted in the last section, we identified a host of avenues that the communities of transformation 

(CoTs) used to expand during the maturing phase. Given that communities of practice (CoPs) often 
tend to be more localized and rarely have the stated goal of expanding widely, the literature on CoPs 
does not have much information about expansion strategies. Our study is the first to document these 
types of expansion strategies for communities of this nature. This section builds on section 5, which 
described outcomes of the CoTs and the broader impacts that they have had over time. The expansion strategies 
outlined in this section provide more detail about how CoTs can have an impact across many spheres—disciplinary, 
national, and even international. 


The communities in our study adopted six foci to spread reforms and promote growth; we have used these foci to 
organize the results sections below. The six foci are disciplinary, institutional, sector-wide (e.g., liberal arts, research uni¬ 
versity, etc.), constituent-based, national, and international, and we present them in this order, expanding outward from 
the most local to the widest scope. This can be visualized like the layers of an onion (see Figure 9.1). All of these foci 
must inform efforts to deepen and spread STEM reform, and we hope that this section assists future STEM reform 
leaders to grapple with them. In Table 9.1, we present a summary of the findings for each focus; within each focus 
we include information related to growth, strategies, leverage points (arising from strengths, capabilities, or history), 
and related challenges. In this section, we describe and document the various approaches to STEM reform in each of 
these areas. The CoTs’ stories of the maturing phase demonstrate how communities decide on strategic approaches 
that build on their emergent strengths. These types of community efforts have not been captured in any other study. 
These collective approaches to learning can also be considered as an expansion of the traditional literature on CoPs, 
offering a new set of growth strategies not found in earlier studies. 

Leaders of each CoT connected their interest in expanding their communities with their aspirations to meet the 
needs of STEM reform, but they reported that they were not aware of the multiple levels or foci from the outset. 
They described how knowing about these foci would have been helpful to their expansion, sustainability, and success, 
which is why we have decided to highlight them in this section. While this section outlines the strategies, leverage 
points, and challenges connected to each focus area, we direct the reader to our paper on the maturing phase (Kezar 
& Gehrke, 2015b) for more detailed analyses. 


Disciplinary Focus 


Two of the communities—the POGIL Project and BioQUEST—used disciplinary or professional societies in their 
approaches to achieving STEM reform. The assumption underlying this strategy is that disciplinary societies strongly 
shape the teaching norms within their respective fields, and that by working through them it is possible to effectively 
alter the approaches to teaching within different fields. There were four main strategies these CoTs used within the 
disciplinary focus: developing textbooks, developing other materials, conducting meetings at disciplinary societies, and ob¬ 
taining grants that were focused on reaching new disciplines. 

The POGIL Project created a partnership with a publisher in order to develop a series of textbooks that use the 
POGIL Project activities for different disciplines. By developing textbooks, they aimed to spread the POGIL Project 
by making the resources and materials readily available, noting the importance of working with publishers that pro¬ 
vide marketing for textbooks. In addition to textbooks and publishing agreements, materials and resources related to 
an innovative teaching technique were also placed on websites (particularly in searchable databases) so that faculty 
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from different disciplines could identify activities and resources to include in their classrooms. For example, Bio- 
QUEST has created an extensive collection of materials that are searchable on their website, including case studies, 
Microsoft Excel activities, and computer simulations, all targeted to specific disciplines and usable to meet the needs 
of various biology or chemistry courses. 

In order to ensure that faculty in various disciplines use these rich resources, both of these communities put together 
presentations regularly at disciplinary conferences and guide people to their textbooks and free online activities and 
materials. Over the years, they have also had booths at disciplinary conferences that describe their materials. How¬ 
ever, in order to continue to create materials to address the needs of various disciplines, grant funding both of these 
communities have relied on grant funding, and they have also sought out grants to develop activities and textbooks as 
they move into new disciplines. One POGIL Project leader described an experience with a subgrant that was pivotal 
to expanding into a new disciplinary area: “One area [in which] we weren’t doing much work was mathematics, and 
now we have a grant to develop the POGIL Project materials for Calculus, which is a huge area on every campus.” 
Individuals involved with these grants worked not only to spread the POGIL Project ideas through the new ma¬ 
terials, but they also often became leaders within their disciplinary communities as a result of involvement in these 
grants. This helped them to spread the word amongst their colleagues. Thus, the grants served a role in terms of creat¬ 
ing new leadership within new disciplines. 

Leadership in these two communities described the challenge of expanding into new disciplinary societies and work¬ 
ing to maintain a presence across various disciplines. Each community had initial success within a single discipline, 
but then struggled due to overextension of time, leadership, and resources, as they attempted to expand or even to 
maintain their presence within that discipline over time. Various leaders talked about the importance of moving in 
new directions only where there is clear leadership and energy: “We recognize that if there is not a critical mass of 
interested faculty, then going in a new direction, even when there is a desire, just won’t pay off. You also have to have 
materials that resonate with that discipline and that is hard before you develop leadership there. So we are more cau¬ 
tious now and really weigh not just is there a need, but are there leaders and people willing to put in lots of time and 
energy into a new area.” 


Institutional Focus 


Two of the communities decided to focus on creating STEM reform through institutions by encouraging the spread 
of practices across science departments within particular campuses. The assumption behind working with institutions 
is that they establish the reward structures and policies to which faculty must respond; therefore, without working 
with institutions, reform is unlikely. The vehicles or strategies for an institutional approach included having teams of 
faculty and administrators attend events, forming institution-based grant projects, utilizing consultancies, supporting general 
or broad curriculum-based projects, and emphasizing leadership development. 

Both SENCER and PKAL have a practice of inviting teams of faculty and administrators to attend their annual 
conferences, events, and symposia. One leader described the way these campus teams were pivotal to their reform 
approach: “One of the things that PKAL pushed is traveling together as a team to events, and I think that that helps 
the local people stay together and develop relationships. You don't hesitate to contact those people when you need 
advice later when back on campus.” Leaders described how changes were unlikely to occur if only individual faculty 
members attended events and were isolated change agents upon their return to their institutions. Administrators 
were included in these teams, to support changes in policies and practices on campus. The events were structured so 
that teams returning to campus could take action to institutionalize the changes. 

Another approach to STEM reform at the institutional level was to obtain grant funding for projects that were 
aimed at change across particular institutions. PKAL obtained grants over several decades that were aimed at in¬ 
stitutionalizing changes through interdisciplinary teaching, active learning, or using more culturally relevant teach- 
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ing approaches; these grants funded groups of institutions (often 12 to 30) in projects aimed to serve as models for 
institutionalizing reform. These projects were designed to help spur change at other institutions. 

Consultancies are a third strategy for creating institutional change and spreading reform. PKAL obtained a grant 
from the Keck foundation to conduct close to 100 institutional consultancies with the goal of moving institutions 
further along in their reform efforts by providing expert advice from experienced practitioners. SENCER established 
what it called “house calls,” in which SENCER leaders would come out to campuses to help faculty and administra¬ 
tors think about ways to “SENCER-ize” their curriculum. 

A fourth strategy for institutional reform is visible in SENCER’s overall institutional approach. SENCER focuses 
on an institution-wide adoption of a pedagogy and approach to science education; it does not aim at changing single 
courses, departments, or even individual faculty, but in altering the overall curriculum of science and general educa¬ 
tion courses. This automatically involves many faculty across the institution, and it is focused on creating broader 
discussion about teaching and learning across departments. 

Both PKAL and SENCER recognize that institutional changes required leadership, particularly leadership built 
among faculty who often lack strategies for implementing change and the skills of persuasion, vision setting, and 
relationship building. As a result, both communities created leadership development activities and programs to help 
cultivate change agents that could institutionalize the changes they were promoting. 

One challenge related to the institutional focus and approach is the task of helping teams to maintain momentum 
when they return to their campuses. This was not always a focus for the communities studied, and it could perhaps 
have increased their impact on the institutional level. An additional challenge related to grant funding is that funders 
generally do not provide enough long-term financial support to foster institutional change; this change takes more 
time than provided by most grant terms. A third, more general challenge observed is that strategies, such as consul¬ 
tancies, were sometimes abandoned when the communities lacked the expertise to maximize them as leverage points 
for institutional change. 


Sector-wide Focus 


Several of these communities began their work with small liberal arts colleges; this is likely the result of the fact 
that much of the STEM reform movement originated with experimentation within this sector. Several of the com¬ 
munities in our study used this history, and the relationships that it left behind, as a base to spread across the sector. 
CoTs can encourage uptake across a significant number of institutions within a sector by leveraging national associa¬ 
tions that work with these sectors and becoming visible in their collective dialogues, communication avenues such 
as newsletters and publications, and events such as annual conferences. The assumptions underlying a sector-based 
strategy are that different institutional types require attention to different needs and that various drivers influence 
sectors in different ways. Among the CoTs in our study, PKAL emerged from a consortium of small liberal arts col¬ 
leges; SENCER gained traction through affiliation with the Association of American Colleges and Universities; and 
BioQUEST and the POGIL Project both started at innovative liberal arts colleges. In general, the focus on sector 
was far less central to strategy, as compared to the disciplinary and institutional foci, which were the primary drivers 
of decision-making. Within this sector-based approach, albeit limited, there were three relatively common strategies 
deployed: partnering with associations , consortia, or groups that represent the sector ; obtaining grants to work with the sector, 
and hosting gatherings for individuals from that sector. 

The first strategy, partnering with associations, consortia, or groups representing the sector, is fairly self-explanatory. 
Because most sectors are represented by national and regional organizations, the communities partnered with these 
organizations to create joint publications, to present at their conferences or meetings, and to provide communications 
about the work of the reform communities and its potential impact within the sector. 
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The second strategy, obtaining grants to work in a new sector, proved effective for several of the communities studied. 
For example, BIOQUEST developed a grant, called “C3 Cyberlearning Project,” that worked directly with commu¬ 
nity colleges to create materials for biology courses that include active learning approaches, embedding the Bio- 
QUEST Three P’s of problem-posing, problem-solving, and peer persuasion. 

Another approach to working with a sector is to host meetings that bring together key leaders from that particu¬ 
lar sector. For example, PKAL gathered private liberal arts presidents and key leaders from time to time to discuss 
STEM reform priorities in the context of the liberal arts mission. Similarly, the POGIL Project drew on its foothold 
in the Mid-Atlantic region by gathering faculty from liberal arts colleges in the area to discuss STEM reform and to 
help spread those ideas. 

There were several challenges related to the sector-based approach. Sector-based grants tended to be temporary, and 
it could become a strain on resources to sustain those funding relationships. One respondent noted, “We have tried 
for years to get more grants for working with our minority-serving colleges, but funders seem to feel: ‘well, we gave 
you that money so you should be all set now.’” Another source of difficulty arose from being solely identified with a 
specific sector when trying to expand the community’s reach. For example, PKAL has been misidentified as exclu¬ 
sively an organization for liberal arts colleges, limiting its impact across sectors. Another challenge for this sector¬ 
wide approach is the task of maintaining the interest of leaders (e.g., presidents) in the sector, as they are often drawn 
in many different directions and have competing priorities. 


Constituent-based Focus 


Two of the communities played active roles in attempting to connect important constituent groups (including stu¬ 
dents, policymakers or legislators, informal educators, and teachers) that they felt helped to enable STEM reform 
efforts and broaden the impact of their work. It is important to describe this area of focus, because the CoTs felt 
that engaging these constituent groups could contribute to reaching reform goals. SENCER, for example, works 
to include student groups in conferences, events, and communications. Their assumption is that reform will require 
outside pressures or resources to be successful, and students bring this perspective. Similar to the other focus areas, 
strategies to reach out to constituent groups draw upon partnerships, hosting meetings, and obtaining grants. Below, we 
describe the various ways the CoTs deployed these strategies with different groups. Each constituent group affected 
is italicized to help illustrate the connective aspect of this approach. 

The POGIL Project leaders referenced students as partners in the process throughout their materials and trainings, 
while SENCER believed that STEM reform efforts should be student-centered and should include the student 
voice. One practitioner told us that, “philosophically, it is important that we included students in events because this 
is ultimately about making learning relevant for students.” Another important reform community is policymakers. 
SENCER created the Washington Symposium and Capitol Hill Poster Session, an annual event that brings faculty, 
administrators, and student leaders to Washington, DC, to present on the individual reform efforts that were going 
on at particular campuses. At the event, poster sessions are presented to congressional staff to show how campuses are 
changing, particularly as they work to solve public policy issues related to sustainability, health care, poverty, etc. The 
goal of this strategy is to gain greater support in terms of grant funding for STEM reform efforts and to encourage 
more state and local support for STEM reform. SENCER also works with informal educators from museums, librar¬ 
ies, and science centers. These constituents have resources and materials that can be utilized to make learning more 
active, experiential, and relevant for students. As a result of this commitment, SENCER obtained a grant to partner 
more actively with informal educators and to host them at conferences. The POGIL Project leaders also actively 
work with high school teachers ; they have obtained several grants to bring their teaching methods into different school 
districts and develop high-school-level the POGIL Project materials and activities. 

Participants expressed several concerns with this constituent-based approach. Faculty involved with reaching out to 
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these new groups worried about whether these efforts added value to the existing community as well as to the new 
groups. Reaching out to new constituents can diffuse the leadership, energy, and resources of the community. There 
are also concerns about whether the CoT has the expertise and materials needed to support the new community 
members, and if new members will be obtaining enough value to stay a part of the community in the long term. A 
faculty member captured these anxieties: “Given this is a new group, not faculty, it’s hard to know if we can provide 
a meaningful community for them. What value do we add? Will they come back? We just don’t know exactly given 
this is not a direction we have gone before.” 


National Focus 


All of the CoTs also worked at the national level to spread their reforms across the country and to embed it within 
state, regional, and federal or national groups with the power to alter the infrastructure of support for STEM reform. 
By operating at this wide, networked level, these communities can connect actors across disciplines, institutions, and 
sectors to enhance STEM reform efforts. In terms of strategies to spread their efforts nationally, STEM reform com¬ 
munity leaders created regional networks, developed other networks or communities, hosted broad stakeholder meetings, and 
participated in national reform efforts. 

A strategy used by all of the CoTs to expand nationally was to establish regional networks or communities. Typi¬ 
cally, each community used grant funding to establish and set up a regional network or several networks and to build 
regional leadership. This was the initial approach used by PKAL, SENCER, and the POGIL Project. Another ap¬ 
proach with a national focus is to host broad stakeholder meetings. PKAL attempted to create a national presence by 
hosting meetings in Washington, DC, with the major national higher education groups at the end of each of its grant 
periods. 

These communities also participated or were otherwise included in national reform projects, and they used this vis¬ 
ibility as a lever to create change. For example, BioQUEST leaders regularly were part of national biology reform 
discussions hosted by the National Academies and other important national groups. This participation resulted in 
major recommendations, such as BIO 2010 (National Research Council, 2003). PKAL was instrumental to the 
creation a variety of different national networks aimed at STEM reform, each more specialized in its mission, such as 
the National Numeracy Network. By creating new STEM reform communities, these CoTs furthered their influence 
and impact nationally in the arena of STEM reform. 

One of the biggest challenges to working on the national level relates to a leadership gap inherent to the iterative 
process. CoTs found difficulty fostering the same level of enthusiasm and leadership in the regional communities, 
as compared to the central community. Another challenge is that communities often devote much time and energy 
to engaging in the national dialogues and connecting with national efforts for STEM reform, sometimes with little 
attention returned to them for their work. This can create a tension for the CoT, between giving time and energy to 
these national efforts and nurturing the core community and its immediate impact. 


International Focus 


A last focus for expanding the work of the CoTs was to explore the possibilities of international outreach. To build 
an international focus, the communities utilize advisory boards and encourage and accept international invitations. 
We did not find well-articulated assumptions underlying the global efforts of these communities at present, but some 
participants suggested that other countries may have ideas that can enrich our own practices, and that the enthusiasm 
around these reforms in some other countries may actually fuel efforts back in the U.S. At the time of our research, 
BioQUEST had the strongest international reach and exposure, and this was a result of including international 
individuals on their advisory boards and making international connections to teaching and learning centers and 
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disciplinary leaders abroad. However, this still constituted a relatively minor emphasis in the community’s work. One 
BioQUEST leader described how this international work developed: “John [the original leader of BioQUEST] had 
a lot of interest in more international issues, and we began to include international leaders on our advisory board and 
also started to get lots of invitations to travel and present our materials. So I think it really started as a result of his 
interest.” But, when that original leader stepped down, there was less interest in international efforts, and the exist¬ 
ing efforts began to dissipate over time. The POGIL Project also has accepted invitations for international visits to 
Australia and New Zealand, but these international connections have not yielded a systemic focus as yet. 

In terms of challenges for international approaches, some participants noted that it can be problematic if this work 
is purely driven by a single leader. As shown in the BioQUEST example, once their leader stepped down there was 
limited interest to continue this work. Therefore, leaders need to have succession plans for international efforts they 
start—especially when the leaders are near retirement. Those trying to reach out internationally also mentioned as 
obstacles the time it takes to travel and the need for in-person connections to make viable CoT connections interna¬ 
tionally. There was discussion about whether there is a way to create branches of the POGIL Project, SENCER, and 
other groups internationally, or if it was better for the home community to simply expand its membership outward 
from the U.S. For all of these groups, these international efforts were still experimental, both in terms of developing 
strategies and in terms of being able to demonstrate impact. This was clearly an area of growth in the future. 


Summary 


In this section, we described the ways in which the communities of transformation in our study expanded their work 
through six different foci—disciplinary, institutional, sector-wide, constituent-based, national, and international—as 
they sought to grow and spread their reforms. Each of these foci contained important strategies for communities to 
expand to reach more faculty and to scale their reforms. There is also much to learn from the challenges that arose for 
work in each of these areas. This work coincided with the maturing phase of the CoTs; in section 10, we turn to the 
stewardship phase and to a model for sustainability of communities, informed by our data. 
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Table 9.1: Summary of Focus Areas, Leverage Points, Strategies, and Related Challenges 


Challenges 


Foe us Area Purpose, Strategies, & 

Leverage Points 


Disciplinary Focus 

Used by the POGIL Project and BioQUEST 

Purpose: Shape overall discipline and 
scale throughout profession 

Strategies: 1. textbooks, curricular mate¬ 
rials; 2. meetings at disciplinary societ¬ 
ies; and, 3. obtaining grants focused on 
reaching new disciplines 

Leverage point: Access to influential 
disciplinary leaders either within com¬ 
munity or through community members 

Lack of expertise to develop materials 
and texts 

Strain to resources if discipline is a sec¬ 
ondary strategy 

Need to maintain disciplinary alliances 
already gained as community presses into 
new disciplines 

Danger that a critical mass emerges 
within discipline too slowly, and too 
many resources are absorbed in effort 

Institutional Focus 

Used by PKAL and SENCER 

Purpose: Shape institutional policy and 
rewards to work on institutionalizing 
changes within institutions 

Strategies: 1. working with groups or 
teams from campuses; 2. institution- 
based grant projects; 3. utilizing consul¬ 
tancies; 4. leadership development; and 5. 
focusing on institution-wide adoption of 
pedagogy 

Leverage points: Partnerships with orga¬ 
nizations connected to institutional lead¬ 
ers on campuses, or institutional leaders 
that are already part of the community 

Need to support teams after they return 
home 

Without an established leadership de¬ 
velopment program, faculty may fail to 
translate these skills onto campus 

Short-term perspective of funders related 
to institutional change strategies 

Faculty leadership of community may 
be inexperienced with working with 
administrators (especially challenging for 
consultancies) 

Sector-wide Focus 

Used by PKAL, the POGIL Project, SENC¬ 
ER, and BioQUEST 

Purpose: Achieve scale by working 
within an entire institutional sector 
(e.g., liberal arts colleges), achievable by 
recognizing and addressing particular 
needs and drivers of influence for differ¬ 
ent sectors 

Strategies: 1. partnerships with associa¬ 
tions, consortia, or groups that represent 
the sector; 2. obtaining grants to work 
with the sector; and, 3. hosting gather¬ 
ings for individuals in that sector 

Leverage points: Have an entree through 
connection to leaders, influential cam¬ 
puses, or other role models in a sector 

Community can become overly identified 
with a particular sector and have trouble 
gaining recognition as relevant for other 
sectors 

Sector grants tend to be one time only, 
and sustaining relationships within a sec¬ 
tor can become a strain 

Sector leaders change priorities regularly, 
and it is hard to capture their interest 
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Foe us Area 


Purpose, Strategies, & 
Leverage Points 


Challenges 


Constituent-based Focus 

Used by SENCER and the POGIL Project 

Purpose: Scale impact by connecting to 
important constituent groups that enable 
STEM reform efforts 

Strategies: 1. partnerships with constitu¬ 
ent groups; 2. hosting meetings; and, 3. 
obtaining grants to support work with 
constituent groups 

Leverage points: Location of community 
leaders in DC for access to policymakers, 
active leadership of students in com¬ 
munity, and demand from a group—i.e., 
teachers—to incite passion in the work 

Difficulty formulating a clear mission to 
work with these groups, often with no 
intangible benefits for community 

Difficulty measuring or communicating 
value added to the constituent groups 
targeted 

Challenge of maintaining connection and 
providing materials to support constitu¬ 
ent group as not the primary audience or 
members within the community 

National Focus 

Used by PKAL, the POGIL Project, SENC¬ 
ER, and BioQUEST 

Purpose: Scale by spreading the reform 
across the country and embedding it 
within state, regional, and federal or 
national groups 

Strategies: 1. creating regional networks; 

2. developing other networks or commu¬ 
nities; 3. hosting broad stakeholder meet¬ 
ings, and, 4. participating in national 
reform efforts 

Leverage points: Concentration of 
community members and leaders in a 
particular region and connection to other 
networks or national efforts 

Difficulty of replicating energy and cul¬ 
ture of national community within local 
and regional areas 

Lack of local leadership or weaker leader¬ 
ship as compared to that which exists at 
national level 

Conflict between building national part¬ 
nership or broad stakeholder meetings 
while maintaining support for the core, 
local work of the community 

International 

Used by BioQUEST and the POGIL Project 

Purpose: Scale impact by reaching out to 
faculty and institutions in other countries 

Strategies: 1. building international 
presence on advisory boards; 2. accepting 
and encouraging national and interna¬ 
tional invitations 

Leverage point: No real evidence of con¬ 
sistent leverage points in our research 

Leadership interest must be continuous 
and not too centralized in order to main¬ 
tain international connections 

Time investment 
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X. Sustainability Model: 

Understanding the Stewardship Phase 


E ach of the four communities of transformation (CoTs) initially started without any sense that it would 
become an ongoing community for faculty-led STEM reform. About five to seven years into their 
existence, each CoT began to realize the importance of its impact and began a trajectory towards 
developing a sustaining model. Viewed through the lens of the model of Wenger et al. (2002; reviewed 
in section 2), the communities were all in the stewardship stage at this time. Each CoT was reacting 
to the challenges noted in that model, and each also faced the additional concern of building infrastructure to create 
sustainability—this latter concern is more closely reflected in the professional learning community literature, also 
described in section 2. The findings of our research describe the important sustainability features that these com¬ 
munities formulated, and how these features enabled the CoTs to continue. Drawing on our findings, we describe 
here a model for sustainability for communities of transformation, with the following features: 1) leadership develop¬ 
ment, distribution, and succession planning; 2) a viable financial model (branding, distinctive identity); 3) a professional 
staff, 4) mechanisms for feedback and advice; 5) research and assessment, and 6) an articulated strategy. Importantly, 
we found that the elements of this model were represented in each CoT in our study, but often developed to varying 
degrees. This serves as a reminder that these CoTs are not ideal cases. Throughout the sections below, we highlight 
some of the challenges in each area that can make difficult the task of striving toward sustainability. 

Figure 10.1 (Page 70) captures the sustainability model that emerged from our study in order to help guide future 
communities. Each area is identified with a particular direction needed to advance sustainability. Feedback helps the 
community to evolve; assessment provides legitimacy and enables funding; professional staff ensures work is conducted 
and supports accountability; leadership creates ideas, maintains energy, and ensures that the culture and philosophy 
of the community is lived out in its practices; a financial model ensures stability for staffing and makes it possible to 
stay focused on impact; and strategy ensures focus critical to maintaining the community identity/mission. There are 
often several avenues available to address each area of the sustainability model. For example, in the area of feedback, 
communities may use advisory boards, self-studies, or surveys; by offering multiple strategies, we hope to give new 
communities the flexibility to find solutions that fit their strengths and goals. We believe that these findings are ap¬ 
plicable to other CoTs inside and outside higher education, specifically other non-organizationally situated commu¬ 
nities of practice. We now review each element of the sustainability model. 


Leadership Development, Distribution, and Succession Planning 


A key component of sustainability is ensuring continuity of leadership through a succession plan. Our findings show 
how leaders in these CoTs provided the enthusiasm, passion, vision, ideas, and public face of the community. Such 
community leaders also helped support the philosophy and mission of their CoTs by embodying and practicing the 
community ideals and values. Even when a CoT had a more distributed leadership structure, participants looked to 
a small number of individuals to epitomize the philosophy, values, and culture of the community. While we saw how 
communities tried to distribute these values among a variety of individuals, sustainability seems to be dependent on 
having some individuals that are “central,” embodying these values and characteristics. In order to support sustain¬ 
ability, a variety of different leadership issues need to be addressed. These include: 1) an ongoing cycle to bring in new 
generations of individuals that may potentially move into leadership positions; 2) the development of individuals that 
are part of the community to play leadership roles; 3) occasional retirement of longstanding leaders, including on ad¬ 
visory boards, in subgroups/communities, and in voluntary roles, in order to allow fresh ideas to emerge in all groups; 
4) distribution of leadership among members; and 5) succession planning. 
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Figure 10.1: Sustainability Model for Non-Organizationally Situated STEM Reform Communities 
of Transformation 
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A Viable Financial Model 


As non-organizationally situated communities of practice, these CoTs depended on grants for support of their work. 
Each CoT experienced periods during which it became difficult to obtain grants, and forcing the community to face 
the possibility of phasing out of existence. Also, the CoTs regularly found themselves stretching their missions to 
obtain funding, thus threatening the identity and shared goals of the community. Stable financing is necessary for 
these communities to avert these crises and continue to have an impact in their areas of reform. This is an issue that 
most community leaders shied away from for a few reasons: as faculty, they typically lacked expertise in business and 
financial planning; their communities had an ethic that supported open access, and it was frowned upon to charge 
for items or services; and, they often did not see the financial aspects as central to community activity. To become 
sustainable, each CoT considered and ultimately selected from among a variety of financial models, including: 1) 
becoming a membership organization; 2) selling materials or resources; 3) charging for events; 4) creating a partner¬ 
ship with another organization; and 5) becoming a nonprofit organization, which often included utilizing a variety of 
the aforementioned strategies. 


A Professional Staff 


The human resources of each CoT typically consisted of a few faculty members funded through a grant and an 
expanded set of volunteers. With some CoTs, the initial leadership was completely voluntary and unpaid, not funded 
from a grant or any other source. Several of the CoTs continued working for years with a shoestring staff—one or 
two individuals funded through grants or shared personnel with other projects. Many participants noted the chal¬ 
lenge of working in this minimal way, and it was captured succinctly by one: “I think that we can do better in terms 
of our processes, procedures, and policies, and this just goes back to the whole conversation we had earlier about hav¬ 
ing internal structures within Project Kaleidoscope that kind of determine how we move forward in a systematized 
way. It's hard to expand when there’s no structure underneath. It's hard to just rely on a grassroots approach and 
expand at the same time.” 

Yet, through success obtaining multiple grants, the expansion of the community, or increasingly ambitious goals, 
these communities evolved to establish a more professional form of human resources. Without the evolution to a 
more professional staff, communities are unable to complete the work designated in their grants successfully or to 
meet their larger community goals. 12 Moreover, a professionalized staff led to sustainability in each case by ensuring 
work completion, by establishing that appropriate expertise exists, and by maintaining accountability. In their mature 
form, the professional staff of these communities ensured that goals were met in order to facilitate further funding 
and maintained the accountability that a volunteer base often was unable to enforce. This is not to say that volunteer 
staffing and leadership is not essential for the success of the CoTs; rather, fully depending on volunteers makes these 
communities vulnerable. 


Formal Feedbackand Advice Mechanism 


Sustaining the CoTs requires systematic feedback about their work. Communities of transformation starting organi¬ 
cally often lacked the practice of obtaining feedback at all, or they received limited feedback; however, this changed 
over time. The most frequently utilized mechanism for feedback was advisory boards, but CoTs also used other forms 
of feedback from their members, such as surveys, formal evaluations, or self-studies. Feedback helps a community to 
evolve and to address challenges, thus fostering sustainability. A community leader described the importance of“be- 


12 As they grow larger, communities of practice can often take on structures that resemble organizations (Wenger et al., 

2002), but they are still distinctively CoPs because of the work they do. 
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ing self-critical and questioning our ways of operating” as one of the most significant parts of sustainability. Another 
community leader noted how feedback that challenged assumptions was important to success and sustainability: 

“And that’s really the way PKAL functioned, it was to keep reflecting on testing, challenging its conception of what 
worked in undergraduate STEM education and then to use that vision to shape a response to whatever was pressing.” 
Feedback can come in many forms, including: 1) advisory boards and/or steering committees; 2) surveys and other 
assessments that get feedback from community members; 3) two-way communication mechanisms, like social media; 
and 4) general reflection, self-questioning, and using a critical eye to reflect on how the community is operating 
and where it is going. The most successful systems for gathering feedback were those that were embedded centrally 
within the guiding philosophy, such as the POGIL Project’s strengths-improvements-insights model, which ensured 
that each event and resource would garner enough feedback to allow for ongoing improvements. 


Assessment/Resea rc h 


Another key component of sustainability is conducting assessment or research to demonstrate the value and im¬ 
pact of the community over time. Demonstrating the impact of the community is significant for continued grant 
funding, but it also functions to demonstrate value as the community transitions to a membership organization or 
attempts to partner or merge with another organization. Any approach for creating a viable financial model also 
depends on demonstrated evidence that the community has added value in connection with its mission, goals, and/ 
or participants. One individual shared the importance of assessment to the community’s long-term sustainability: “I 
don’t think we had any idea early on that the assessment would help with funding or creating opportunities to pres¬ 
ent and expand our membership, but it has done that, and I see how it’s one of the key factors in our sustainability. 

I mean it makes sense that you can demonstrate that the ideas actually work.” A more formal feedback mechanism, 
as described above, typically buttressed efforts to conduct assessment and research. These two facets of sustainability 
are inter-related, as CoTs can capitalize on processes for receiving feedback to develop additional and more in-depth 
assessment. 


A Community-Derived Strategy that is Articulated, yet Evolving 


Over time, each community recognized that it had a particular niche and expertise that was of interest to others. The 
CoTs used this understanding to center or focus their work; their work, however, was not always focused on develop¬ 
ing this niche. At times, individual leaders decided on new directions that were outside their communities’known 
strengths; this led to diffusion of efforts that was symptomatic of a lack of strategic direction within the CoT. As the 
communities matured and moved toward sustainability, they recognized the importance of a strategic focus. An artic¬ 
ulated strategy typically took the form of a strategic plan for the community, but it might also be a strategy document 
that is less formal. While planning typically was focused on developing the specialty or focus area of the CoT, leaders 
also had to create a plan for the community itself, addressing questions, such as: How do we continue to bring in new 
members? How do we continue to engage members with different levels of expertise? How do we build leadership? 
This is how the notion of strategy connects to leadership and staffing. A community leader noted that, “you have to 
plan for community as well: How do we keep the enthusiasm? How do we bring in new people? And how do we 
have enough critical mass? This needs to be part of the strategy as well.” Creating a strategy provided an opportunity 
for CoTs to examine external factors that might be important for the future—for example, changes in funders’ grant 
directions or new technologies. While the communities were always scanning their environments, a narrow group of 
leaders might do so selectively, based only on their particular interests. A wider strategic planning process allowed for 
a much broader community involvement, introducing a more representative set of external concerns to be brought 
into strategy development. 
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Articulating a strategy also helps the communities to hone their direction. Many spent time drifting along, doing the 
work required by the most recently funded grant rather than identifying work that was best suited for the specific 
community and its niche. For example, PKAL identified that its perceived strength was in leadership development, 
and it began to focus activities more in this area to draw on its well-known expertise. The POGIL Project identi¬ 
fied that it needed to expand from a narrow pedagogical strategy (i.e., process-oriented guided inquiry learning) to a 
broader one (i.e., active learning) that better encapsulated the work of the community and would position it to reach 
many more faculty members. BioQUEST had long focused on simulations and complex technological interventions, 
but its leaders realized that they could move to simpler technologies, such as Excel spreadsheets, and still engage 
their philosophical approach in a way that was strategic. 


Summary 


As communities of transformation reach the stewardship phase in their evolution, issues and decisions of sustainabil¬ 
ity must be addressed. In this section, we described a model for sustainability encapsulating six key factors—leader¬ 
ship, funding, assessment, professional staff, feedback, and strategy—to inform future communities as they seek to 
continue their work for extended periods of time. Our findings related to sustainability, as well as those highlighted 
throughout this report, suggest areas for future consideration for these communities and others like them. We now 
turn to such considerations in section 11, where we explore ideas that could contribute to the success and longevity of 
these and other such communities in the future. 
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XI. Future Co nside ratio ns for STEM Reform 
Communities of Transformation 


T he communities of transformation (CoTs) in our study have had an important impact on STEM 
reform, and they have increased their impact by expanding into multiple areas, such as disciplinary 
societies, curricular efforts, institutional change efforts, sector discussions, international efforts, and 
powerful constituent groups. This section is about additional areas that we identified that such com¬ 
munities might expand into in the future to increase their impact. The items identified below are 
largely drawn from our interviews with leaders across these four CoTs. They represent future directions for such com¬ 
munities to continue to evolve their work. 


Capitalize More on Disciplinary Work 


Two of the networks we studied had strong connections to a set of disciplinary societies, but many of the faculty and 
administrators we spoke with believed that more could be done to broaden and deepen the impact in this area. There 
was recognition that disciplinary societies are beginning to support teaching, not just research. With large-scale 
initiatives, such as the Association of American Universities’ efforts to gain greater recognition for teaching, there 
may be greater opportunity than in the past to garner the attention of disciplinary leaders in order to support STEM 
reform. CoTs can work to change the dialogue and professional standards within disciplines to include evidence- 
based teaching practices. These CoTs are well positioned to push for change in the conversations about teaching in 
the disciplines. Our data suggest that one of the main avenues for faculty members to become involved with CoTs 
is through attending presentations at disciplinary conferences. Given that this is already a major way that CoTs are 
involving and engaging members, leaders within these communities might better coordinate and strategize ways to 
create synergy within such presentations in order to have greater impact over time. In addition, leaders within these 
STEM reform communities could help to establish committees or task forces focused on evidence-based teaching 
practices within various disciplinary societies. It is important to recognize that STEM reform is more likely to hap¬ 
pen when multiple stakeholder groups within the overall enterprise are aligned with a similar message, such as “use 
more evidence-based teaching practices.” 


Explore Complementary Online/Virtual Ways to FosterCommunity 


Each of these communities has experimented with various forms of technology to connect their members, particu¬ 
larly through social media. However, they reported that they had only mixed success with engaging faculty members. 
The demographic data from our survey showed that the faculties involved in these communities skew towards higher 
seniority. Over time, we imagine that efforts to get more junior faculty involved will result in increasing demand for 
technological ways to connect. As technology continues to evolve in ways that become more personal, it will become 
easier to ensure that the philosophical aspects of these communities—which members find so engaging—are main¬ 
tained and communicated through technological means. An example to consider is the Center for the Integration 
of Research, Teaching and Learning (CIRTL) Network’s online cafe course. Other technological platforms can be 
helpful in this area, including Adobe Connect, GoToMeeting, and Blackboard Collaborate (all virtual, video-based 
meeting technologies). 


Focus More on Network Development 


The CoTs in our study focus on community and relationship development to provide mentorship and advice for 


74 I Communities ofTransformation 









members. They do some work to connect faculty across institutions, but they do not have intentional plans or struc¬ 
tures for network development. When these communities tried to use regional networks for this purpose, they all 
struggled to get those networks off the ground. Although the regional networks were only partially successful, it is 
important to foster networking among members by setting up interest groups within the communities or by actively 
connecting faculty within similar institutions or disciplines. Although we think these communities were successful 
overall because of their intentional nurturing, as opposed to the more informal structure of networks, we do think 
that fostering informal networks within these communities could enhance and complement their work. Such net¬ 
works could increase the potential for innovation to spread through the communities’ membership (Rogers, 2003; 
Valente, 1995). More attention to the networking potential within these groups might yield additional outcomes and 
even greater impact and spread of ideas. 


Work with Graduate Students'Align with Graduate Initiatives 


None of these communities do substantial work with graduate students. While they continue to connect with faculty, 
it would likely benefit these communities to connect with graduate-focused efforts aimed at using evidence-based 
teaching, such as the Center for the Integration of Research, Teaching and Learning (CIRTL). CIRTL aims to work 
with graduate students to help them approach their teaching as a scholarly endeavor. The same graduate students 
who participate in such programs will be looking for communities to support their work once they become faculty. 
CIRTL has now expanded to include 26 universities, reaching the institutions that train most of the doctoral stu¬ 
dents nationally. As things currently stand, graduate students lack an introduction to the communities of transforma¬ 
tion that can continue to support their work once they have received their degrees and begin their faculty careers. 

We recommend that there be greater communication between STEM reform communities and graduate student 
initiatives aimed at using evidence-based teaching. Graduate students may also be easier to work with on such reform 
effort, since they do not have deeply ingrained habits or allegiances to the status quo. 


Work with CentersforTeaching and Learning on Campus 


Most campuses have a center for teaching and learning (CTL) that is aimed at helping faculty to improve their 
practice. However, most of the STEM reform CoTs in our study do not have connections to the directors of these 
centers. Furthermore, they do not work with these units to recruit members or to help CoT participants to become 
aware of resources for support once they return to their home institutions after events. There are several advantages 
to working more directly with CTLs. First, they can help alleviate the isolation that many faculty feel when they go 
back to campus; directors of CTLs can help connect faculty with other innovators on campus. Second, by connecting 
many individuals who are interested in STEM reform, there are possibilities for CTLs to be a part of broader change 
efforts across a department or college. Some CTLs have also been successful setting up learning communities related 
to STEM reform. If CTL directors knew about the cadres of faculty members receiving training in these issues from 
CoTs, they would be more likely to set up such groups and garner institutional support for such efforts. 


Work with Postdoctoral Organizations 


There is a growing number of postdoctoral fellows across the country, particularly in the sciences. As more individu¬ 
als move into these positions, they become another group of potential new faculty that should be mentored and 
trained in evidence-based teaching practices. Thus, they should become familiar with the work of the STEM reform 
communities. The National Postdoctoral Association would be an important group for communities to connect with 
in the future. 
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Expand Consultation Work 


Three of the STEM reform communities have been involved in some form of consultation with campuses interested 
in more broadly institutionalizing STEM reform. While this approach was used for a short period of time, few of 
the STEM reform communities maintained a consulting corps to help spread and support change. None of them 
stopped using this approach because it lacked merit; we recommend that it be reconsidered by these STEM reform 
communities and adopted by future communities. This model is sustainable because individual campuses pay for 
the consultations, and there is money in administrative budgets to support such work. None of these STEM reform 
communities set up a formalized consulting corps and publicized it nationally. This could be an important resource 
for broader reform. 


Create On-Campus Learning Communities'Communities of Transformation 


Once faculty return to their campuses, they are often alone or a part of a small circle of innovators. One of the ways 
that the national CoTs could support local efforts is by educating individuals at their workshops and events on how 
to establish learning communities/CoTs back on their home campuses. Learning communities are becoming bet¬ 
ter known nationally, as there is research to support their efficacy in helping faculty to adopt new teaching practices. 
Learning communities can be a type of community of transformation, but they are typically aimed at meeting for a 
narrower, circumscribed period of time (12 to 18 months) and around a particular topic (e.g., service learning). The 
nationally based CoTs in our study could create workshops aimed at helping local campus leaders to develop learning 
communities, thus supporting faculty members as they return home to their campuses. Such workshops could draw 
upon the growing body of literature about learning communities and how they can best support faculty learning (see 
Cox, 2003; 2004). 


Supporting Innova torson their Home Campuses 


Another way for leaders in these CoTs to help support innovators at home is to write letters to academic leaders 
informing them of faculty members’ involvement in evidence-based teaching practices. Communities can also create 
awards and fellowships to honor individuals, and they can make sure that campus leaders at participants’ home insti¬ 
tutions are aware of these recognitions. While this sometimes occurred in the CoTs we studied, it was an underuti¬ 
lized strategy for change. A more systemic way for these STEM reform communities to foster legitimation back on 
the individual home campuses will also help to seed further reform. Over time, it is important that public perceptions 
of the work shift; what is currently seen as innovative activity should simply become viewed as normatively strong 
teaching practices. Influencing the views of academic leaders on college and university campuses is an important part 
of this strategy. 


Consider Approaches that Deeply Embed STEM Reform 


It is advantageous to embed changes deeply, so that they do not fluctuate with shifts in leadership. One of the most 
intriguing strategies for doing this is exemplified by PKAL’s work with facilities, which structurally embeds new 
teaching practices into the institution by reshaping the architecture of the setting in which learning takes place. Such 
efforts aim to change the underlying structures that prevent evidence-based teaching practices; this work is not only 
supportive of engaged pedagogy, but it builds reform into the institution in a way that has strong potential for persis¬ 
tent impact and expanded scale. 
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Capitalize on Other Reform Initiatives 


Much of the work of STEM reform is isolated from the many other institutional learning reform efforts. Most 
campuses have some form of pedagogical or curricular reform efforts that are ongoing. STEM reform efforts tend 
to happen in isolation of these other initiatives. The reform efforts of the CoTs can be more successful when they 
align with the broader educational reform efforts unfolding within an institution, whether those efforts are located in 
general education, student support, or curricular redesign. CoTs can help faculty to identify these other reform efforts 
on their campuses and help them to connect in order to support their own work. 


Consider Becoming a Membership Organization 


Our study was able to identify several ways that the CoTs were able to sustain themselves over time (see section 10). 
One model that we think holds particular promise for future consideration is to become a membership organization. 
The Council for Undergraduate Research is an example of a community that evolved into a membership organiza¬ 
tion, now sustained by thousands of members. The size and scale of the reform CoTs in our study puts them in a 
position to consider this strategy to ensure long-term success and viability. 


Consider a Networked Improvement Community 


The Carnegie Foundation has supported reform work using a networked improvement community model in higher 
education. In this model, faculty and administrators work in a network of institutions aimed at similar goals. They 
collect and share data about improved STEM practices, create professional development activities to embed practices, 
and try to benefit from the power of multiple institutions working together for change. The Bayview Alliance—an¬ 
other STEM reform effort—uses the networked improvement community model, as well. CoTs might create similar 
networks among the faculty with whom they work, capitalizing on the fact that their membership spans literally 
hundreds of institutions, many of which have already prioritized efforts aimed at change. By leveraging their wide 
reach and trying to create more depth through network improvement communities, they might expand their goals of 
institutional change. Networked improvement communities are aimed specifically at changes at the organizational or 
institutional level, offering a unique possibility to expand the reach of CoTs at that level of impact. 


Additional Mechanisms to Support Systemic Change 


The strength of these CoTs is their ability to create buy-in and motivation for faculty to improve practices in support 
of STEM student success. They also help to provide individuals with a community of support in the face of depart¬ 
mental and even institutional opposition. We found that, as faculty move into formal roles as department chairs and 
deans, they also can help create change to the broader system. Particular change efforts typically do not work at all 
levels of the system at once; the mechanisms present in CoTs work particularly well on the ground to create indi¬ 
vidual consciousness change. However, we believe that CoTs could do more to effect systemic change at the level of 
incentives, promotion and tenure, and disciplinary norms. Such efforts would greatly enhance the success CoTs have 
already had in helping faculty members to see their potential and to become empowered to influence their institu¬ 
tions from the ground up. Certainly we have evidence that these CoTs have already had impact on this broader sys¬ 
tem, but we believe that they can be designed to do even more. This is the motivation behind the recommendations 
above, and we hope that increased work establishing on-campus learning communities, connecting with centers for 
teaching and learning, and training academic leaders will help to alter incentive systems and create new institutional 
priorities and culture. 


We now turn to future considerations for research in order to expand our knowledge of CoTs and their work for 
STEM reform. 
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XII. AneasforFurtherResearch 


O ur inquiry into these four communities of transformation (CoTs) reveals several areas in need of 
further research, particularly as we seek to understand the phenomenon of STEM education com¬ 
munity formation and the task of scaling up reform. This section outlines several areas that would 
benefit further inquiry 


The Continuum from Community of Practice to Social Networks 


We entered this study anticipating that some of the entities that were our subjects might reflect a network approach, 
and others might be more appropriately understood as communities of practice (CoPs). Through our research, 
however, we found that all four of the entities were best understood as communities—in particular, communities of 
transformation (CoTs). While the entities that we studied did not turn out to be networks, we believe that there may 
be reform efforts appropriate to some settings that could emerge from a more passive, networked approach. Such 
an approach would be defined by more informal interactions, loose structure, and a focus on fostering relationships 
based on common disciplines or research interests, rather than on nurturing communities like those in our study. We 
believe that future research should be aimed at identifying other variations of reform communities, particularly ones 
that act like social networks, and the challenges and opportunities of those groups. In the end, our study was able to 
focus on the benefits, design, and lifecycle of CoTs, and another study could do the same for networks. 


Communities of Transformation 


Through this study, we identified a new variant of communities of practice, which we called communities of transfor¬ 
mation. Since we did not go into the study with the purpose of identifying and documenting the structure of a CoT, 
we feel that it is important that further research be conducted to better understand it as a new variant of a CoP. 

As outlined in section 4, we identified three primary features of CoTs: 1. Formation and documentation of a phi¬ 
losophy, which is an innovative approach and radical departure from existing practice; 2. Living integration of the 
philosophy throughout events and activities, modeled by the leadership of the organization; and 3. A community that 
supports the new practice once an individual returns to a status quo community. We identified some unique aspects 
present in how these entities are formed (importance of formulating a philosophy), how they sustain themselves 
(ways that the philosophy is embedded into the design), and their outcomes (a network of change agents that helps 
create and sustain change in status quo spaces). 

One of the important outcomes of these CoTs is that they are able to create deep changes related to STEM reform; 
for example, they challenge the underlying assumptions about being a scientist and approaches to conducting science. 
Such deep philosophical issues are often not embedded in the evidence-based teaching practices for which these 
communities advocate, but they emerge organically from the philosophical orientation of the work of the communi¬ 
ties. Advocates for gender parity and improved success of minorities in STEM have identified these types of deep 
changes as central to the project of rethinking science and making it inclusive of broader groups of people. Much 
work is still needed to fully understand how these communities of transformation can contribute to the philosophi¬ 
cal changes needed to make science more equitable. We imagine that future research can look at challenges related to 
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the philosophical side of these communities; for example, can it lead to philosophical divides, dogmatism, or lack of 
flexibility? 

We also found that CoTs are not always intentional about preparing people for returning to the particular status quo 
cultures present on home campuses. For example, some faculty participants return to campuses that are supportive of 
innovative pedagogies, but the faculty are not necessarily prepared to maximize their impact in these settings. There¬ 
fore, it is important to look at the ways that these communities provide potentially different types of support for 
individuals that are returning to such different environments. 

In addition, our study only examined CoTs that were non-organizationally located. However, this is not a necessary 
feature of such communities. Future research should seek to identify CoTs that may exist within organizations in or¬ 
der to examine their abilities to engage community members in deeper forms of engagement. Do such organization- 
based communities exist? Can they exist? 


Social Networks within Communities of Transformation 


In section 2, we reviewed the literature on social networks because the communities in our study clearly foster rela¬ 
tionships among their members. However, our study did not focus on understanding the networks of relationships in 
these communities; rather, we focused more on participant engagement within the community and the ways in which 
community design can influence outcomes. Future research should examine the extent to which the informal net¬ 
works of personal relationships formed in these communities contribute to spreading the reform strategies. Members 
of the POGIL Project leadership have begun this work, examining how their network of workshop facilitators has 
grown and their approach has spread to different parts of the United States. Social network analysis is a useful ana¬ 
lytical tool to identify the ways in which community connections are made and grow, as well as to study the spread of 
adoption of these STEM reform strategies over time. 


Non-organizationally Located Communities of Practice 


Because the entities in our study are either not organizationally located or began outside of organizations, they 
face some specific challenges related to sustainability; we were able to identify many of these challenges in section 
10. However, we did not go into the study considering the non-organizational character of these communities as a 
central influence on their design or outcomes. After seeing how the non-organizational character of the CoTs shaped 
sustainability, we wonder about its impact on these other areas. For example, how can the right level of mentoring 
be maintained within a loosely organized group? Perhaps there needs to be a balance between in-person and virtual 
means of connection in order to achieve outcomes in these non-organizationally located communities of practice. 

We did not specifically study these types of issues. Future studies should continue to explore the impact of not being 
organizationally located on the way these entities operate and on their ability to meet their STEM reform objectives. 


Broader Impacts 


Our survey was focused on individual, departmental, and institutional impacts from participation in the STEM 
reform communities. We initially had questions related to broader impacts, beyond the institution, but our advisory 
board suggested that we remove these questions to avoid cognitive overload on the individuals taking the survey and 
to go deeper into the impacts that were at the localized level. In addition, they advised us that individual members of 
the community, the target of the survey, would be less likely than leaders within the communities to identify broader 
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impacts. We appreciated the advisory board’s advice, and we believe this was justified and important feedback. 
However, we also found that there are many broader impacts that these communities of transformation were able to 
contribute, identified in our interviews but not reflected in the surveys. We believe that continued research to identify 
these broader impacts will help us to understand the contribution of these communities in the long term. Currently, 
we have qualitative data about broader impacts, but further study is necessary to ascertain whether these findings are 
generalizable. 


OtherApproachesto Examining Outcomes 


It is challenging to examine outcomes in broad-ranging CoTs such as those we studied. Because of the scale 
of these communities, we do not have the ability to observe changes in practice or to follow departmental 
changes—approaches that would be feasible in the case of a local CoP. Logistically, self-report data was one of our 
only viable options. One approach to measuring outcomes would be to examine institutional data from institutions 
at which faculty who belong to these communities work. Such research could examine if there have been noticeable 
improvements in objective measures, such as retention and persistence in STEM disciplines or changes to the num¬ 
ber of STEM majors. Outcomes assessment could also be enhanced by identifying comparable institutions at which 
these strategies and practices are not employed, thus creating a benchmark for institutions active in these CoTs. 
Another approach for understanding departmental and institutional outcomes would be to survey deans or depart¬ 
ment chairs at a random selection of campuses involved. Even then, however, it would be a challenge to obtain a large 
sample from those campuses; surveys among academic leaders typically have low response rates. Thus, we suggest that 
future research focus on a subset of institutions with faculty involved in these communities. Research can then try 
to document changes in individual faculty, departments, and the institution broadly to try to evaluate the impact of 
these communities. 


Comparison of Different Models of STEM Reform Efforts 


Approaches to STEM reform differ in terms of their targets: some focus on disciplinary societies; others focus on 
institutional change (such as the Association of American Universities projects); some emphasize networked ap¬ 
proaches (such as the Bayview Alliance networked-community approach); and others focus on individual faculty 
(such as the CoTs in this study). There have been few studies that examine and compare these various approaches to 
STEM reform, and we lack data on whether some work better for certain purposes or reasons. For example, we know 
from our research that the CoTs we studied are particularly important for faculty who are isolated on their campuses 
and do not have other reformers to work with. However, individuals who reported having a lot of support on their 
campuses told us they were less likely to be deeply involved with these CoTs and to have ongoing involvement. There 
might also be ways that various approaches and strategies for STEM reform can be aligned to support one another. 
Research that compares these different approaches could seek out these types of alignments and synergies. 


The Tension between Stability/Sustainability and Appeal of Informal 
Community 


We identified a model for long-term sustainability for these groups. We did wonder if, as they became more 
organizational and began to be more formal, the communities might lose the very features that faculty found most 
engaging. For example, participants identified the informal peer-to-peer work, in which ideas could be debated, as 
one of the most attractive aspects of these communities. Yet, as the CoTs moved in the direction of increased stability, 
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we could see that they lost some these types of features. Future work should examine these groups in their later, more 
formalized state to identify the impact on faculty engagement and outcomes associated with moving into a more 
sustainable, long-term model. 


The Challenges of Expansion 


Our research identified key avenues for expansion, leverage points to gauge which directions to expand into, and 
challenges of expansion (see section 9). While we unearthed some solutions to those challenges, we did not find 
practicable strategies to be used in every area. Future research could explore additional ways communities can over¬ 
come the challenges we identified. 


The Effect of the Changing Faculty on STEM Reform 


All of these communities expressed concern about the changes in the academic workforce, and particularly about the 
ways that these changes may impact their ability to connect with faculty. As faculty are increasingly off the tenure 
track, overburdened, and under-supported, how can communities initiate and sustain reform? It is important that 
future research explore the impact of the changing faculty on efforts to improve STEM education in general, and in 
particular among these CoTs. 
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XIII. Conclusion 


s we entered this study, we were interested in understanding ways to scale efforts to improve STEM 
education. We leave the study with an appreciation that communities of transformation have and 
continue to play a central role by providing communities of support for individual faculty. Through 
participation in these communities, we saw lone innovators transformed into individuals who have 
the support, legitimacy, and camaraderie to continue their work to improve STEM within their own 
classrooms. We also identified how these communities support, although to a lesser extent, departmental and institu¬ 
tional changes as well. We charted ways that they are expanding their impact into disciplinary societies, into national 
groups and organizations, and even internationally. In this report, we have provided advice gleaned from our research 
to help communities expand their role in departmental, institutional, and even enterprise-level change. 

By working on both individual and systems-level change at the same time, these communities play a unique role 
nationally in supporting the improvement of STEM. While their efforts currently are more effective at helping in¬ 
dividuals, there is the capacity for these communities to grow to leverage increased change at the systemic level. Our 
study documented some of the systems-level changes taking place through these communities and several approaches 

that can be leveraged toward further impact in the future. Our research 
and reported findings are designed to help encourage work in this direc¬ 
tion and to help provide a pathway for other communities that want to 
play a role in systems change. 

However, it is important to note that, in some environments, a more 
systemic approach to change is not currently possible. On some cam¬ 
puses and in some environments, starting at a more local level, with a 
small number of faculty, may be the most viable approach. We also have 
evidence that the spread of good practices relies on support and connec¬ 
tivity offered to isolated innovators within status quo departments and institutions. The strategies of communities of 
transformation, namely empowering individual faculty members through enthusiasm and philosophical engagement, 
works from the ground up to complement other efforts to change reward structures or disciplinary values. As we 
consider institutional change in general, and STEM reform in particular, too often do we polarize our approaches be¬ 
tween those aimed at individuals and those focusing on the system. Both of these approaches are required for change, 
and these communities of transformation offer a bridge between them. This is an important role that needs support 
in order to develop to its full capacity for impact in the future. 


By working on both individual 
and systems-level change at the 
same time , these communities 
play a unique role nationally in 
supporting the improvement of 
STEM. 
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